Papers with large language model
PreAct: Prediction Enhances Agent’s Planning Ability (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to analyze Markov decision processes (MDPs) are based on chain-of-thought (COT) and historical thought, action, and observation. |
| Approach: | They propose a model that integrates prediction, reasoning, and action with other models to provide a wider range of reasoning and more efficient actions. |
| Outcome: | The proposed model outperforms the ReAct method in completing complex tasks and is more efficient when paired with other memory or selection strategy techniques. |
Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classification (2025.acl-short)
Copied to clipboard
| Challenge: | Current intent classification systems face significant challenges due to the vast number of possible intents and significant semantic overlap among similar intent classes. |
| Approach: | They propose a dynamic label refinement method that retrieves relevant examples for a test input and leverages a large language model to dynamically refine intent labels based on semantic understanding. |
| Outcome: | The proposed method resolves confusion between semantically similar intents and generates more interpretable intent labels. |
General Collaborative Framework between Large Language Model and Experts for Universal Information Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing unified information extraction approaches face challenges such as noise interference, abstract label semantics, and diverse span granularity. |
| Approach: | They propose a general Collaborative Information Extraction framework to address these challenges in universal information extraction tasks. |
| Outcome: | The proposed framework is based on a general Recognizer and task-specific Experts for recognizing predefined types and extracting spans respectively. |
Pro-QuEST: A Prompt-chain based Quiz Engine for testing Specialized Technical Product Knowledge (2026.eacl-demo)
Copied to clipboard
| Challenge: | Specialized benchmarks can be leveraged to create quizzes that can effectively train engineering and marketing personnel on novel product offerings in a continually growing Cisco product space. |
| Approach: | They propose to generate multiple-choice questions using domain-specific prompts using a set of professional certification textbooks and a range of latest open-source and proprietary LLMs. |
| Outcome: | The proposed quiz engine generates multiple-choice questions using domain-specific prompts and a range of latest open-source, and proprietary LLMs. |
Large Language Models as Realistic Microservice Trace Generators (2025.emnlp-main)
Copied to clipboard
| Challenge: | Obtaining real-world traces is difficult due to limited public data availability and the difficulty of collecting them at large scale from diverse environments. |
| Approach: | They propose to train a large language model to generate microservice call graphs using a recursive approach to capture hierarchical structures and implicit constraints in such traces. |
| Outcome: | The proposed method outperforms existing methods in accuracy and validity. |
Explainable CED: A Dataset for Explainable Critical Error Detection in Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing studies of critical error detection lack content addressing the causes of catastrophic errors. |
| Approach: | They propose a dataset that introduces the attributes of error explanation and correction regarding critical errors. |
| Outcome: | The proposed dataset reduces time costs and mitigates human annotation bias. |
Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to forecast news media responses have limited exploration of how to best process and utilize these important features. |
| Approach: | They propose a framework that leverages a large language model to induce a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics. |
| Outcome: | The proposed framework surpasses state-of-the-art in experimental evaluations for both zero-shot and supervised settings, demonstrating its effectiveness in response forecasting. |
PersonaTrace: Synthesizing Realistic Digital Footprints with LLM Agents (2026.eacl-industry)
Copied to clipboard
Minjia Wang, Yunfeng Wang, Xiao Ma, Dexin Lv, Qifan Guo, Lynn Zheng, Benliang Wang, Lei Wang, Jiannan Li, Yongwei Xing, Junzhe Xu, Zheng Sun
| Challenge: | Publicly available corpora cover only slivers of human activity, such as email threads, chat logs, purchase histories, sensor traces, and provide large-scale supervision for data-hungry machine-learning pipelines. |
| Approach: | They propose a method for synthesizing realistic digital footprints using large language model agents from a structured user profile. |
| Outcome: | The proposed method generates diverse sequences of user events, producing corresponding digital artifacts such as emails, messages, calendar entries, reminders, etc. |
Doc-React: Multi-page Heterogeneous Document Question-answering (2025.acl-short)
Copied to clipboard
Junda Wu, Yu Xia, Tong Yu, Xiang Chen, Sai Sree Harsha, Akash V Maharaj, Ruiyi Zhang, Victor Bursztyn, Sungchul Kim, Ryan A. Rossi, Julian McAuley, Yunyao Li, Ritwik Sinha
| Challenge: | Existing methods for integrating information across multiple modalities are suboptimal for multi-page, multimodal documents. |
| Approach: | They propose an adaptive iterative framework that balances information gain and uncertainty reduction at each step. |
| Outcome: | The proposed framework captures relevant multimodal content and achieves strong performance on complex QA tasks. |
EmpathyEar: An Open-source Avatar Multimodal Empathetic Chatbot (2024.acl-demos)
Copied to clipboard
| Challenge: | EmpathyEar is an open-source, avatar-based multimodal empathetic chatbot . currently, ERG systems rely on text, sound, and vision . |
| Approach: | They propose an open-source, avatar-based multimodal empathetic chatbot to fill the gap in traditional text-only ERG systems. |
| Outcome: | The proposed system enables users to generate emotional responses to user queries . it can also generate avatars with talking faces and synchronized speeches . |
Generative Reviewer Agents: Scalable Simulacra of Peer Review (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing peer review mechanisms are limited by the small fraction of researchers with established networks. |
| Approach: | They propose a system that extends a large language model and equips agents with reviewer personas derived from historical data to enable generative reviewers. |
| Outcome: | The proposed architecture performs comparable to human reviewers in providing detailed feedback and predicting paper outcomes. |
PiKGL: Leveraging Pruned Knowledge Graphs for Explainable Stance Detection (2026.tacl-1)
Copied to clipboard
Bingbing Wang, Jingjie Lin, Zhixin Bai, Xintong Song, Qianlong Wang, Min Yang, Xi Zeng, Jing Li, Ruifeng Xu
| Challenge: | Experimental results demonstrate that a Pruned interpretable knowledge Graph Learning framework for explainable stance detection is state-of-the-art for social media stance prediction. |
| Approach: | They propose a Pruned interpretable knowledge Graph Learning framework for explainable stance detection that incorporates commonsense knowledge and prunes redundant information to ensure precision and minimize noise. |
| Outcome: | The proposed framework achieves state-of-the-art on three public datasets. |
Tulun: Transparent and Adaptable Low-resource Machine Translation (2025.acl-demo)
Copied to clipboard
| Challenge: | a low-resource language that is the lingua franca in Timor-Leste lacks available corpora in the health domain. |
| Approach: | They propose a solution that combines neural MT with large language model-based post-editing guided by existing glossaries and translation memories. |
| Outcome: | The proposed system outperforms both standalone MT and LLM approaches across six low-resource languages on the FLORES dataset. |
Improving Retrieval-Augmented Generation without Taxonomy-based Error Categorization (2026.acl-short)
Copied to clipboard
| Challenge: | Recent work implicitly assumes reliable critic feedback and focuses on planning strategies, while paying limited attention to the robustness of the error correction process itself. |
| Approach: | They propose a response-action learning paradigm that maps flawed RAG outputs to error-mitigating action plans without explicit criticism. |
| Outcome: | The proposed model improves the factual accuracy of large language model outputs without explicit error categorization. |
RankGen: Improving Text Generation with Large Ranking Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Modern language models assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix. |
| Approach: | They propose a 1.2B parameter encoder model for English that scores model generations given a prefix. |
| Outcome: | The proposed model outperforms decoding algorithms on automatic metrics and human evaluations with English writers. |
Beyond IVR: Benchmarking Customer Support LLM Agents for Business-Adherence (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing benchmarks focus on tool usage or task completion, overlooking an agent’s capacity to adhere to multi-step policies, navigate task dependencies, and remain robust to unpredictable user or environment behavior. |
| Approach: | They propose a benchmark to assess policy-aware agents in customer support using a dynamic-prompt agent and a static-promped agent that explicitly models policy control. |
| Outcome: | The proposed benchmark assesses agent's ability to adhere to multi-step policies, navigate task dependencies, and remain robust to unpredictable user or environment behavior. |
Token Level Routing Inference System for Edge Devices (2025.acl-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been gaining in performance but deployment in edge devices faces significant hurdles due to their high computational complexity. |
| Approach: | They propose a collaborative decoding system that allows small models to perform on-device inference while selectively consulting a cloud-based large model for critical token generation. |
| Outcome: | The proposed system achieves 60% performance gain on CommonsenseQA using a 0.5B model on an M1 MacBook, with under 7% of tokens generation uploaded to the large model in the cloud. |
FPT: Feature Prompt Tuning for Few-shot Readability Assessment (2024.naacl-long)
Copied to clipboard
| Challenge: | Prompt-based methods lack crucial linguistic knowledge for readability assessment tasks such as word length, sentence length, and usage of different difficulty-level words. |
| Approach: | They propose a new prompt-based tuning framework that incorporates linguistic knowledge and a loss function to calibrate the similarity ranking order between categories. |
| Outcome: | The proposed framework outperforms the large language model gpt-3.5-turbo-16k in most cases. |
Alligators All Around: Mitigating Lexical Confusion in Low-resource Machine Translation (2025.naacl-short)
Copied to clipboard
| Challenge: | Current machine translation systems for low-resource languages have a particular failure mode: they tend to confuse words within a domain. |
| Approach: | They propose a recall-based metric to measure the failure mode of machine translation systems for low-resource languages. |
| Outcome: | The proposed model outperforms a lexicon-based translator in 122 low-resource languages. |
ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing frameworks for large language model (LLM) inference on CPUs overlook overhead of cross-NUMA memory access. |
| Approach: | They propose a lightweight LLM inference architecture designed from the ground up for many-core CPUs. |
| Outcome: | Experimental results show that ArcLight surpasses the performance ceiling of mainstream frameworks, achieving up to 46% higher inference throughput. |
Aligning Sentence Simplification with ESL Learner’s Proficiency for Language Acquisition (2025.naacl-long)
Copied to clipboard
| Challenge: | Text simplification is crucial for improving accessibility and comprehension for English as a Second Language (ESL) learners. |
| Approach: | They propose to simplify complex sentences to appropriate levels while also increasing vocabulary coverage of the target level. |
| Outcome: | The proposed method can increase frequency and diversity of vocabulary of the target level by more than 20% compared to baseline models, while maintaining high simplification quality. |
Japanese-English Sentence Translation Exercises Dataset for Automatic Grading (2024.eacl-srw)
Copied to clipboard
| Challenge: | Sentence translation exercises (STEs) are used in the early stages of L2 language learning. |
| Approach: | They propose to formalize the task of automatic assessment of Sentence Translation Exercises (STEs) they use a dataset of 21 questions and 3,498 student responses to test the framework . |
| Outcome: | The proposed task is based on a dataset of 21,498 student responses between Japanese and English and compares them to baseline models with fine-tuned BERT and GPT-3.5 with few-shot learning. |
sudoLLM: On Multi-role Alignment of Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a framework that allows users to control access rights has not been extensively studied in the large language model realm. |
| Approach: | They propose a framework that allows users to control access rights in a multi-role manner. |
| Outcome: | The proposed framework improves alignment, generalization and resistance to prefix-based jailbreaking attacks. |
Paraphrasing with Large Language Models (D19-56)
Copied to clipboard
| Challenge: | Recent work has shown large language models are adept at text generation and fine-tuning for downstream NLP tasks. |
| Approach: | They propose a system that generates paraphrased examples in autoregressive fashion using a neural network without the need for techniques such as top-k word selection or beam search. |
| Outcome: | The proposed system generates paraphrased examples in autoregressive fashion without the need for techniques such as top-k word selection or beam search. |
LLM as Entity Disambiguator for Biomedical Entity-Linking (2025.acl-short)
Copied to clipboard
| Challenge: | Entity linking involves normalizing a mention in medical text to a unique identifier in a knowledge base, such as UMLS or MeSH. |
| Approach: | They propose to use a large language model as an entity disambiguator to enhance the accuracy of alias-matching entity linking methods. |
| Outcome: | The proposed method surpasses existing methods on biomedical datasets by up to 16 points in accuracy. |
Two Heads Are Better Than One: Audio-Visual Speech Error Correction with Dual Hypotheses (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances have introduced GER frameworks that utilize LLMs to refine ASR outputs. |
| Approach: | They propose a framework that allows a large language model to compose independent N-best hypotheses from separate automatic speech recognition (ASR) and visual speech recognition models. |
| Outcome: | The proposed framework achieves 57.7% error rate gain over standard ASR baseline, compared to single-stream approaches that achieve only 10% gain. |
How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior (2026.acl-long)
Copied to clipboard
Zidi Xiong, Yuping Lin, Wenya Xie, Pengfei He, Zirui Liu, Jiliang Tang, Himabindu Lakkaraju, Zhen Xiang
| Challenge: | In practice, memory designs vary widely across agents due to their diverse objectives and functionalities. |
| Approach: | They conduct an empirical study on how memory management choices impact the LLM agents’ behavior, especially their long-term performance. |
| Outcome: | The proposed methods show that LLM agents display an experience-following property, which results in highly similar agent outputs. |
Constructing a Japanese Rap Lyric Generation Model with GRPO (2026.acl-srw)
Copied to clipboard
| Challenge: | Rap is a vocal style rooted in Hip-Hop culture, characterized by producing rhymes in synchrony with a rhythmic beat. |
| Approach: | They propose a method for generating Japanese rap lyrics with a large language model . the model's rhyming behavior is improved by using existing Japanese rhapsodysts as training data. |
| Outcome: | The proposed method improves outputs that receive moderate or high human ratings on rhyme-related criteria. |
Rethinking Data Mixing from the Perspective of Large Language Models (2026.acl-short)
Copied to clipboard
Yuanjian Xu, Tianze Sun, Changwei Xu, XinLong Zhao, Jianing Hao, Ran Chen, Yang Liu, Ruijie Xu, Stephen Chen, Guang Zhang
| Challenge: | Existing methods to mix data with LLMs have relied on domain definitions derived from intuition. |
| Approach: | They propose a reweighting framework that restructures data scheduling as a graph-constrained optimization problem. |
| Outcome: | The proposed framework achieves competitive performance on GPT-2 models. |
Variable Layerwise Quantization: A Simple and Effective Approach to Quantize LLMs (2025.findings-acl)
Copied to clipboard
Razvan-Gabriel Dumitru, Vikas Yadav, Rishabh Maheshwary, Paul Ioan Clotan, Sathwik Tejaswi Madhusudhan, Mihai Surdeanu
| Challenge: | a meta quantization approach quantizes different layers of a large language model at different bit levels. |
| Approach: | They propose a meta quantization approach that quantizes different layers of a large language model at different bit levels. |
| Outcome: | The proposed method quantizes the most important layers to higher bit precision and less important layers at lower bits. |
HotelMatch-LLM: Joint Multi-Task Training of Small and Large Language Models for Efficient Multimodal Hotel Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | a novel multimodal dense retrieval model for the travel domain addresses limitations of traditional search engines. |
| Approach: | They propose a multimodal dense retrieval model that enables natural language property search . they propose combining a small language model and a large language model for embedding hotel data . |
| Outcome: | The proposed model outperforms state-of-the-art models on four diverse test sets . it is generalizable across LLM architectures and scalability for processing large image galleries . |
Towards Practical and Knowledgeable LLMs for a Multilingual World: A Thesis Proposal (2025.naacl-srw)
Copied to clipboard
| Challenge: | a proposed thesis examines the role that multilinguality occupies in the development of practical and knowledgeable LLMs. |
| Approach: | They propose to use multilingual knowledge to improve LLM performance on NLP tasks . they extend the territorial disputes benchmark to retrieval-augmented generation setting . |
| Outcome: | The proposed methods improve LLMs' performance on standard natural language processing tasks by leveraging their existing multilingual knowledge. |
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward (2025.naacl-long)
Copied to clipboard
Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander G Hauptmann, Yonatan Bisk, Yiming Yang
| Challenge: | Existing studies have demonstrated that direct preference optimization (DPO) can be effective in generalizing large language models, but its effectiveness in video domain remains limited. |
| Approach: | They propose a framework that utilizes detailed video captions as a proxy of video content to enable language models to incorporate this information as supporting evidence for scoring video Question Answering (QA) predictions. |
| Outcome: | The proposed framework shows that it can be used to align language models with video content and improves performance on open-ended video QA tasks. |
Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI (2025.coling-industry)
Copied to clipboard
Yuya Asano, Sabit Hassan, Paras Sharma, Anthony B. Sicilia, Katherine Atwell, Diane Litman, Malihe Alikhani
| Challenge: | Existing ASR correction methods rely on prior user data or named entities . Existing methods based on prior data are not available for goal-oriented dialogues . |
| Approach: | They propose a method that integrates contextual information from the dialogue states of a goal-oriented conversational AI and its tasks into a large language model. |
| Outcome: | The proposed method improves recall and F1 of correction by 34% and 16% while maintaining precision and false positive rate. |
AUTOSUMM: A Comprehensive Framework for LLM-Based Conversation Summarization (2025.acl-industry)
Copied to clipboard
Abhinav Gupta, Devendra Singh, Greig A Cowan, N Kadhiresan, Siddharth Srivastava, Yagneswaran Sriraja, Yoages Kumar Mantri
| Challenge: | Large language models (LLMs) are used to summarize large volumes of textual information into a smaller, more manageable size. |
| Approach: | They propose a large language model-based summarization system for regulated banking environments that generates accurate, privacy-compliant summaries of customer-advisor conversations. |
| Outcome: | The proposed system achieves 94% factual consistency rate and significant reduction in hallucination rate. |
KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Empirical evaluation shows that our approach yields superior performance in both standard task metrics and large language model (LLM)-based evaluation. |
| Approach: | They propose a K-step return estimation method for reinforcement learning (RL)-based knowledge distillation in text generation tasks using the Bellman Optimality Equation. |
| Outcome: | The proposed method performs better on standard task metrics and large language model evaluations on three text generation tasks. |
MA-COIR: Leveraging Semantic Search Index and Generative Models for Ontology-Driven Biomedical Concept Recognition (2025.acl-srw)
Copied to clipboard
Shanshan Liu, Noriki Nishida, Rumana Ferdous Munne, Narumi Tokunaga, Yuki Yamagata, Kouji Kozaki, Yuji Matsumoto
| Challenge: | Existing concepts recognition methods that rely on explicit mention identification fail to capture complex concepts not explicitly stated in the text. |
| Approach: | They propose a framework that reformulates concept recognition as an indexing-recognition task. |
| Outcome: | The proposed framework reduces computational requirements and improves recognition efficiency in low-resource settings. |
Common Sense or Ableism? Rethinking Commonsense Reasoning Through the Lens of Disability (2026.eacl-short)
Copied to clipboard
| Challenge: | a recent study finds that commonsense reasoning is not always universal and can leave disabled people behind . a case study of disabled people with long COVID shows that common sense is not universal . |
| Approach: | They investigate how datasets and models deal with disability in commonsense reasoning . they use annotations from disabled and non-disabled persons for ableism . |
| Outcome: | The proposed datasets have low sensitivity to human-detected ableism but still detect 5 to 25% of entries as ableist. |
TestAgent: An Adaptive and Intelligent Expert for Human Assessment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing adaptive testing methods face several challenges due to mechanized nature of most algorithms and noisy response data. |
| Approach: | They propose to use large language models to enhance adaptive testing through interactive engagement to capture test-takers’ responses and anomalies. |
| Outcome: | The proposed agent achieves more accurate results with 20% fewer questions than state-of-the-art baselines and testers preferred it in speed, smoothness, and other dimensions. |
Gardener: An Agentic AI System for Single-Cell RNA Sequence Analysis (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing large language models encode workflow progress as conversational state and rely on cloud-centric execution, which hinders traceability and auditability. |
| Approach: | Gardener is an open-source desktop application for macOS and windows under the Apache License 2.0. |
| Outcome: | Gardener is released as an open-source desktop application for macOS and Windows under the Apache License 2.0. |
AutoTrain: No-code training for state-of-the-art models (2024.emnlp-demo)
Copied to clipboard
| Challenge: | AutoTrain is an open-source, no code tool/library which can be used to train models on custom datasets. |
| Approach: | They propose an open-source, no-code tool/library to train models on custom datasets. |
| Outcome: | The open-source, no-code tool/library can be used to train models on custom datasets. |
KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for visual and language understanding are not capable of multimodal reasoning. |
| Approach: | They propose a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts. |
| Outcome: | The proposed model performs state-of-the-art on the Visual Commonsense Generation task. |
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)
Copied to clipboard
Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, Xiaoyin Che, Zhiyuan Liu, Maosong Sun
| Challenge: | Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation. |
| Approach: | They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation. |
| Outcome: | The proposed framework generates high-quality documentation for the entire project. |
Conceptual structure coheres in human cognition but not in large language models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Neural network models of language have long been used to develop hypotheses about conceptual representation in the mind and brain. |
| Approach: | They propose to use three techniques borrowed from cognitive psychology to estimate lexical-semantic structure in humans and a large language model. |
| Outcome: | The proposed models show that human-like models can estimate lexical-semantic structure robustly to cultural, language, and method of estimation. |
Time Matters: An End-to-End Solution for Temporal Claim Verification (2024.emnlp-industry)
Copied to clipboard
| Challenge: | despite growing interest, the verification of temporal claims has not received much attention in the digital space. |
| Approach: | They propose an end-to-end solution that takes cues of temporal information and extracts relevant evidence sentences from temporal claims. |
| Outcome: | The proposed method significantly improves accuracy of temporal claim verification. |
FLIQA-AD: a Fusion Model with Large Language Model for Better Diagnose and MMSE Prediction of Alzheimer’s Disease (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing classification and regression models that only extract finer-grained information from magnetic resonance imaging (MRI) may not be effective for Alzheimer's disease (AD). |
| Approach: | They propose to use a 3D Adapter in a Vision Transformer to extract the patient's EHR information and questions related to the disease as text prompts. |
| Outcome: | The proposed model can discriminate and predict the corresponding MMSE score based on the extracted brain structural information and textual content . |
Granite Guardian: Comprehensive LLM Safeguarding (2025.naacl-industry)
Copied to clipboard
Inkit Padhi, Manish Nagireddy, Giandomenico Cornacchia, Subhajit Chaudhury, Tejaswini Pedapati, Pierre Dognin, Keerthiram Murugesan, Erik Miehling, Martín Santillán Cooper, Kieran Fraser, Giulio Zizzo, Muhammad Zaid Hameed, Mark Purcell, Michael Desmond, Qian Pan, Inge Vejsbjerg, Elizabeth M. Daly, Michael Hind, Werner Geyer, Ambrish Rawat, Kush R. Varshney, Prasanna Sattigeri
| Challenge: | a suite of advanced models is designed to detect and mitigate risks associated with prompts and responses. |
| Approach: | a team of researchers develop a model family to detect and mitigate risks associated with prompts and responses. the model family is based on the Granite 3.0 language models. |
| Outcome: | a new model family is designed to detect and mitigate risks associated with prompts and responses. |
Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds (2022.findings-emnlp)
Copied to clipboard
| Challenge: | e.g., stew skillet, swamp squash) are not fully compositional, but highly predictable based on whether the modifier and head refer to artifacts or natural kinds. |
| Approach: | They propose to compare the interpretations of novel English noun compounds with the large language model GPT-3, which is governed by interpretive principles. |
| Outcome: | The results show that the large language model GPT-3 reasoning only about specific lexical items is consistent with the Levin et al.'s theory. |
Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies explore performing text style transfer on attributes like age, gender, formality, politeness, and formality. |
| Approach: | They propose a framework that freezes the pre-trained model’s original parameters and enables the development of a multiple-attribute text style transfer model. |
| Outcome: | The proposed model outperforms state-of-the-art models on sentiment transfer and multiple-attribute transfer tasks with significantly less computational resources. |
Leveraging Variation Theory in Counterfactual Data Augmentation for Optimized Active Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Active Learning (AL) allows users to provide focused annotations to integrate human preferences and domain knowledge into machine learning models. |
| Approach: | They propose a counterfactual data augmentation approach inspired by Variation Theory to generate targeted variations along key conceptual dimensions. |
| Outcome: | The proposed approach achieves significantly higher performance when there are fewer annotated data, showing it can address the cold start problem in Active Learning. |
DaCoM: Strategies to Construct Domain-specific Low-resource Language Machine Translation Dataset (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing models for low-resource languages struggle with domain-specific terms and lack of expert annotators for dataset creation. |
| Approach: | They propose a method for collecting low-resource language pairs from industrial domains using a large language model and neural machine translation framework. |
| Outcome: | The proposed model performs poorly on DaCoM-created datasets with up to 53.7 BLEURT points difference depending on domain inclusion. |
Automatic Model Selection with Large Language Models for Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-Thought and Program-Aided Language Models offer different strengths and weaknesses. |
| Approach: | They propose a model selection method that uses a large language model to select between two different reasoning methods. |
| Outcome: | The proposed method shows significant performance improvements across eight reasoning datasets with Codex, ChatGPT, and GPT-4. |
From Superficial to Deep: Integrating External Knowledge for Follow-up Question Generation Using Knowledge Graph and LLM (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for generating follow-up questions are limited to shallow contextual questions that are uninspiring and have a large gap to the human level. |
| Approach: | They propose a three-stage external knowledge-enhanced follow-up question generation method which generates questions by identifying contextual topics, building a knowledge graph online, and finally combining these with a large language model to generate the final question. |
| Outcome: | The proposed method generates questions by identifying contextual topics, building a knowledge graph (KG) online, and finally combining these with a large language model to generate the final question. |
Enhancing Online Recruitment with Category-Aware MoE and LLM-based Data Augmentation (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing methods to measure the matching degree of a job and a candidate face several challenges, such as low-quality job descriptions and similar candidate-job pairs. |
| Approach: | They propose a large language model-based method that polishes and rewrites low-quality job descriptions by leveraging chain-of-thought prompts and category-aware Mixture of Experts (MoE) module incorporates category embeddings to dynamically assign weights to the experts and learns more distinguishable patterns for similar candidate-job pairs. |
| Outcome: | The proposed method surpasses existing methods by 2.40% in AUC and 7.46% in GAUC and boosts click-through conversion rate (CTCVR) by 19.4% in online tests, saving millions of CNY in external headhunting expenses. |
Implanting LLM’s Knowledge via Reading Comprehension Tree for Toxicity Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for toxic content detection are small language model (SLM) based and large language model(LLM) -based. |
| Approach: | They propose to implant LLM's knowledge into SLM based methods to stick to both types of models' strengths by constructing a reading comprehension tree to transfer knowledge between two models. |
| Outcome: | The proposed method can stick to both types of models' strengths . it is compared with existing methods on real-world and machine-generated datasets. |
DatawiseAgent: A Notebook-Centric LLM Agent Framework for Adaptive and Robust Data Science Automation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language model (LLM) agents for data science automation are limited by narrow task scopes, limited generalization across tasks and models, and over-reliance on state-of-the-art (SOTA) LLMs. |
| Approach: | They propose a notebook-centric LLM agent framework for adaptive and robust data science automation. |
| Outcome: | The proposed framework surpasses baselines such as AutoGen and TaskWeaver in performance tests across diverse data science scenarios and models. |
Cross-Modal Projection in Multimodal LLMs Doesn’t Really Project Visual Attributes to Textual Space (2024.acl-short)
Copied to clipboard
| Challenge: | Existing multimodal large language models are limited to general-purpose multimodal tasks like question-answering on natural images. |
| Approach: | They propose to use cross-modal projection networks and a large language model to model domain-specific visual attributes of MLLMs. |
| Outcome: | The proposed models gain domain-specific visual capabilities when the projection is fine-tuned, but the updates do not extract relevant domain-specific visual attributes. |
RE-FIN: Retrieval-based Enrichment for Financial data (2025.coling-industry)
Copied to clipboard
| Challenge: | Financial sentiment analysis (FSA) is a powerful tool to support business decision-making and perform financial forecasting. |
| Approach: | They propose a system that retrieves information from a knowledge base to enrich financial sentences, making them more knowledge-dense and explicit. |
| Outcome: | The proposed system generates propositions from the knowledge base and employs Retrieval-Augmented Generation (RAG) to augment the original text with relevant information. |
TruthSplit: Revealing Conditional Validity in Arguments Through Multi-Worldview Comparative Reasoning (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing argumentation tools analyze properties of the argument, but leave perspective-specific background knowledge implicit. |
| Approach: | They propose a system that analyzes arguments from multiple perspectives using a perspective-dependent analysis approach. |
| Outcome: | The proposed system analyzes arguments from multiple perspectives and generates interpretations, value conflicts and assumption gaps. |
Metamo: Empowering Large Language Models with Psychological Distortion Detection for Cognition-aware Coaching (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Metamo is a browser-based dialogue system that transforms an off-the-shelf large language model into an empathetic coach for everyday workplace concerns. |
| Approach: | They propose a browser-based dialogue system that first identifies the cognitive distortion behind an emotion, then recognizes the user’s emotion, and finally produces a question-centered reply that invites reflection. |
| Outcome: | Empirical tests on public corpora showed that the proposed system improved emotionrecognition quality and response diversity without sacrificing latency. |
What the Router Sees Matters: Funnel Pooling for Fast, Content Driven Expert Routing (2026.eacl-srw)
Copied to clipboard
| Challenge: | Modern large language model (LLM) systems often route inputs to specialized experts to improve accuracy, efficiency, and robustness. |
| Approach: | They propose a lightweight trainable in-model readout that constructs the routing vector directly from token-level hidden states. |
| Outcome: | The proposed model reduces language and source-dataset driven clustering and results in more topic-aligned domains. |
What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representations (2026.acl-short)
Copied to clipboard
Yujie Luo, Zhuoyun Yu, Xuehai Wang, Yuqi Zhu, Ningyu Zhang, Lanning Wei, Lun Du, Da Zheng, Huajun Chen
| Challenge: | Existing approaches to replicate AI research are limited by insufficient background knowledge and the limitations of retrieval-augmented generation methods. |
| Approach: | They propose a pluggable, paper-centric knowledge base that integrates code snippets and technical insights extracted from scientific literature into a verifiable, executable representation. |
| Outcome: | The proposed knowledge base shows significant performance gains on paperBench when integrated into three agent frameworks with two different LLMs. |
ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning (2026.acl-demo)
Copied to clipboard
Vladislav Smirnov, Quang-Chieu Nguyen, Sergey Senichev, Minh Ngoc Ta, Ekaterina Fadeeva, Artem Vazhentsev, Daria Galimzianova, Nikolai Rozanov, Viktor Mazanov, Jingwei Ni, Tianyi Wu, Igor Kiselev, Mrinmaya Sachan, Iryna Gurevych, Preslav Nakov, Timothy Baldwin, Artem Shelmanov
| Challenge: | Existing TTC scaling strategies and reasoning scorers are fragmented and evaluated under inconsistent protocols. |
| Approach: | They propose a framework for seamless test-time compute scaling of large language model reasoning . they use a modular Python library to implement state-of-the-art scaling strategy and scorer families . |
| Outcome: | The proposed framework evaluates performance and computational efficiency on mathematical and coding tasks. |
Bottom-Up Synthesis of Knowledge-Grounded Task-Oriented Dialogues with Iteratively Self-Refined Prompts (2025.naacl-short)
Copied to clipboard
| Challenge: | Training conversational question-answering systems requires in-domain data, which is often scarce in practice. |
| Approach: | They propose a bottom-up approach where QA pairs are generated first and combined into a coherent dialogue. |
| Outcome: | The proposed approach produces more realistic and higher-quality dialogues compared to top-down methods. |
New Compendium of a Myriad of Plants: A New Dataset Describing Ancient Chinese Plants (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to digitize ancient Chinese texts and extract information from them are shallow and inconsistent with modern realities. |
| Approach: | They propose to expand ancient Chinese datasets using large language model . they focus on Great Compendium of Myriad Flowers, an ancient plants dataset . |
| Outcome: | The proposed model can extract plant-related information from classical Chinese poetry and prose. |
Cross-Refine: Improving Natural Language Explanation Generation by Learning in Tandem (2025.coling-main)
Copied to clipboard
| Challenge: | Natural language explanations (NLEs) are vital for elucidating the reasoning behind large language model (LLM) decisions. |
| Approach: | They propose a role-modeling approach that employs two LLMs as generator and critic to generate and refine NLEs. |
| Outcome: | The proposed model outperforms self-refine and can perform with less powerful LLMs. |
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)
Copied to clipboard
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models (2024.emnlp-main)
Copied to clipboard
Ranchi Zhao, Zhen Thai, Yifan Zhang, Shengding Hu, Jie Zhou, Yunqi Ba, Jie Cai, Zhiyuan Liu, Maosong Sun
| Challenge: | Large Language Models (LLMs) are pre-trained on vast datasets composed of billions of tokens harvested from diverse text sources. |
| Approach: | They propose a data engineering method to refine the pretraining corpus through data rating, tagging and editing. |
| Outcome: | The proposed method improves the quality of the pretraining corpus by enhancing 100 billion tokens of the training corpus. |
From Schema to State: Zero-Shot Scheme-Only Dialogue State Tracking via Diverse Synthetic Dialogue and Step-by-Step Distillation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing research classifies zero-shot, scheme-only DST into two main types: the cross-domain scenario and the zero-schemaonly setting. |
| Approach: | They propose a zero-shot, scheme-only approach that generates synthetic dialogues that balance diversity with schema alignment and distills knowledge from a large language model into a smaller model. |
| Outcome: | The proposed approach achieves state-of-the-art performance under zero-shot, scheme-only situation and generalizes effectively to few-shot scenarios. |
Transcending Scaling Laws with 0.1% Extra Compute (2023.emnlp-main)
Copied to clipboard
Yi Tay, Jason Wei, Hyung Chung, Vinh Tran, David So, Siamak Shakeri, Xavier Garcia, Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc Le, Mostafa Dehghani
| Challenge: | Existing scaling of language models is expensive and requires significant computational costs. |
| Approach: | They propose a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
| Outcome: | The proposed method significantly improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
An Empirical Study of Synthetic Data Generation for Implicit Discourse Relation Recognition (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to recognize implicit discourse relations are limited by the lack of training data. |
| Approach: | They propose a method to generate synthetic data for IDRR using a large language model . they extract confused discourse relation pairs based on false negative rate and use two-stage prompting to generate effective synthetic data. |
| Outcome: | The proposed method achieves state-of-the-art macro-F1 performance without sacrificing micro-F1. |
End-to-End Beam Retrieval for Multi-Hop Question Answering (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing beam retrieval frameworks for multi-hop question answering were customized for two-hop questions and were poorly supervised. |
| Approach: | They propose an end-to-end beam retrieval framework for multi-hop question answering . they combine an encoder and two classification heads to optimize the retrieval process . |
| Outcome: | The proposed framework improves on MuSiQue-Ans and surpasses all previous retrievers on HotpotQA and achieves 99.9% precision on 2WikiMultiHopQA. |
AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset (2025.acl-long)
Copied to clipboard
Charles Nimo, Tobi Olatunji, Abraham Toluwase Owodunni, Tassallah Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni, Ezinwanne C. Aka, Folafunmi Omofoye, Foutse Yuehgoh, Timothy Faniran, Bonaventure F. P. Dossou, Moshood O. Yekini, Jonas Kemp, Katherine A Heller, Jude Chidubem Omeke, Chidi Asuzu Md, Naome A Etori, Aïmérou Ndiaye, Ifeoma Okoh, Evans Doe Ocansey, Wendy Kinara, Michael L. Best, Irfan Essa, Stephen Edward Moore, Chris Fourie, Mercy Nyamewaa Asiedu
| Challenge: | Recent advances in large language models (LLMs) performance on medical multiplechoice question (MCQ) benchmarks have stimulated interest from healthcare providers and patients globally. |
| Approach: | They introduce AfriMed-QA, the first largescale Pan-African English multi-specialty medical Question-Answering (QA) dataset, with 15,000 questions sourced from over 60 medical schools across 16 countries. |
| Outcome: | The proposed model outperforms other models in the medical field and is compared with other models. |
Leveraging Generative Large Language Models with Visual Instruction and Demonstration Retrieval for Multimodal Sarcasm Detection (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for multimodal sarcasm detection do not fully utilize cross-modal features, limiting their performance on in-domain datasets. |
| Approach: | They propose a multimodal sarcasm detection model with a designed instruction template and a demonstration retrieval module. |
| Outcome: | The proposed model outperforms existing methods on in-domain datasets and achieves state-of-the-art performance. |
Direct Judgement Preference Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing judge models are largely trained with supervised finetuning on small data scales to perform limited types of evaluation tasks, limiting generalization. |
| Approach: | They propose to train judge models at large data scales with direct preference optimization . they use four training tasks to form three types of preference pairs targeting different aspects of evaluation . |
| Outcome: | The proposed model outperforms GPT-4o and other similar models on 13 benchmarks. |
VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods focus on object hallucinations, focusing on object outputs . current evaluation methods struggle to address subtle semantic distinctions between outputs and reference data . |
| Approach: | They propose a multi-dimensional benchmark covering objects, attributes, and relations . they propose metric that generalizes CHAIR metric and incorporates faithfulness and coverage . |
| Outcome: | The proposed evaluation framework is more comprehensive and better correlated with humans than existing evaluation methods. |
Domain Regeneration: How well do LLMs match syntactic properties of text domains? (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent improvements in large language models have improved their ability to approximate distributions . authors find that LLMs can suffer from model collapse due to domain considerations based on pretraining . |
| Approach: | They use open source LLMs to regenerate permissively licensed English text from Wikipedia and news text. |
| Outcome: | The proposed model can faithfully match the human-generated distributions in a semantically-controlled setting. |
REARANK: Reasoning Re-ranking Agent via Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | REARANK is a large language model-based listwise reasoning reranking agent . it explicitly reasons be- fore reranked results, significantly improving performance and interpretability. |
| Approach: | They propose a large language model-based listwise reasoning reranking agent that explicitly reasons be- fore reranked lists. |
| Outcome: | The proposed agent outperforms GPT-4 on reasoning-intensive benchmarks and surpasses GPL-4 on BRIGHT benchmarks. |
INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent (2025.acl-long)
Copied to clipboard
Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.p. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow, Qianqian Xie
| Challenge: | Recent advances have underscored the potential of large language model (LLM)-based agents in financial decision-making. |
| Approach: | They propose to evaluate LLM agents using 13 different LLMs as backbone models across various market environments and tasks. |
| Outcome: | The proposed framework assesses the reasoning and decision-making capabilities of 13 different LLMs across various market environments and tasks. |
OpenGraph: Towards Open Graph Foundation Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) have emerged as promising techniques for encoding structural information and improving performance in tasks like link prediction and node classification. |
| Approach: | They propose a graph foundation model that generalizes to unseen graph data with different properties. |
| Outcome: | The proposed model achieves remarkable zero-shot graph learning performance across various settings. |
Into the Unknown: Generating Geospatial Descriptions for New Environments (2024.findings-acl)
Copied to clipboard
| Challenge: | Similar to vision-and-language navigation tasks, the Rendezvous (RVS) task requires reasoning over allocentric spatial relationships using non-sequential navigation instructions and maps. |
| Approach: | They propose a large-scale augmentation method for generating high-quality synthetic data for new environments using readily available geospatial data. |
| Outcome: | The proposed method improves accuracy on unseen and seen environments by 45.83% on the Rendezvous (RVS) task. |
DETQUS: Decomposition-Enhanced Transformers for QUery-focused Summarization (2025.naacl-long)
Copied to clipboard
Yasir Khan, Xinlei Wu, Sangpil Youm, Justin Ho, Aryaan Mehboob Shaikh, Jairo Garciga, Rohan Sharma, Bonnie J Dorr
| Challenge: | Query-focused tabular summarization is an emerging task in table-to-text generation . traditional transformer-based approaches face challenges due to token limitations and the complexity of reasoning over large tables. |
| Approach: | They propose a system that leverages tabular decomposition alongside a fine-tuned encoder-decoder model to improve summarization accuracy. |
| Outcome: | a new system outperforms the state-of-the-art REFACTOR model in a Query-focused tabular summarization task . the proposed system achieves a ROUGE-L score of 0.4437, outperforming the previous state- of-the art model . |
Multi-Value-Product Retrieval-Augmented Generation for Industrial Product Attribute Value Identification (2025.emnlp-industry)
Copied to clipboard
Huike Zou, Haiyang Yang, Yindu Su, Chen Li Yu, Qinye Xie, Chengbao Lian, Qingheng Zhang, Shuguang Han, Fei Huang, Jufeng Chen
| Challenge: | Existing methods for product attribute value identification suffer from cascading errors and lack of generalization capability. |
| Approach: | They propose a multi-level retrieval scheme that uses products and attribute values as distinct hierarchical levels in PAVI domain. |
| Outcome: | The proposed method performs better than the state-of-the-art methods on a real-world industrial dataset. |
Beyond Surface-Level Pattern Trap: LLM Agents for Faster and Smarter Cross-Architecture Code Migration (2026.findings-acl)
Copied to clipboard
| Challenge: | cross-architecture code migration is a resource-intensive and errorprone task. |
| Approach: | a framework for cross-architecture code migration is proposed to decouple implementation details through functional mining and code refactoring. |
| Outcome: | a new framework improves performance and correctness over state-of-the-art frameworks on OpenCV migration tasks. |
Multi-component Causal Tracing in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are prone to various forms of safety risks, such as learning and propagating societal biases and even creating harmful or deceptive content through jailbreak attacks. |
| Approach: | They propose a framework for causally tracing multiple components simultaneously that systematically identifies the subsets of components most critical to a desired performance metric. |
| Outcome: | The proposed method outperforms existing methods in identifying components critical to a desired performance metric. |
Globalizing BERT-based Transformer Architectures for Long Document Summarization (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune a large language model on downstream tasks show several limitations when the target task requires to reason with long documents. |
| Approach: | They propose a hierarchical approach where the input is divided in multiple blocks independently processed by the scaled dot-attentions and combined between the successive layers. |
| Outcome: | The proposed approach performs well on three extractive summarization corpora of scientific papers and news articles. |
Self-Specialization: Uncovering Latent Expertise within Large Language Models (2024.findings-acl)
Copied to clipboard
Junmo Kang, Hongyin Luo, Yada Zhu, Jacob Hansen, James Glass, David Cox, Alan Ritter, Rogerio Feris, Leonid Karlinsky
| Challenge: | Recent studies have demonstrated the effectiveness of self-alignment in which a large language model is aligned to follow general instructions using instructional data generated from the model itself. |
| Approach: | They propose to use human-written seeds to align large language models to follow general instructions to achieve cross-task generalization. |
| Outcome: | The proposed model outperforms base models and models that are generally instruction-tuned or have been adapted to the target domain by a large margin. |
FIRE: Fact-checking with Iterative Retrieval and Verification (2025.findings-naacl)
Copied to clipboard
Zhuohan Xie, Rui Xing, Yuxia Wang, Jiahui Geng, Hasan Iqbal, Dhruv Sahnan, Iryna Gurevych, Preslav Nakov
| Challenge: | Fact-checking long-form text is challenging, and breaking it down into multiple atomic claims is not cost-effective. |
| Approach: | They propose a novel agent-based framework that integrates evidence retrieval and claim verification in an iterative manner. |
| Outcome: | The proposed framework reduces large language model (LLM) costs by an average of 7.6 times and search costs by 16.5 times while retaining the same performance. |
A Unified Taxonomy-Guided Instruction Tuning Framework for Entity Set Expansion and Taxonomy Expansion (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies view entity set expansion, taxonomy expansion, and seed-guided taxonomies as three separate tasks. |
| Approach: | They propose a taxonomy-guided instruction tuning framework to teach a large language model to generate siblings and parents for query entities. |
| Outcome: | The proposed framework outperforms baselines on multiple benchmark datasets. |
Multimodal Document-level Triple Extraction via Dynamic Graph Enhancement and Relation-Aware Reflection (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for extracting structured triples knowledge from multimodal documents face limitations in simultaneously processing long textual content and multiple associated images for triple extraction. |
| Approach: | They propose a multimodal document-level triple extraction framework that integrates multimodal text and visual content into a large language model and injects the global information and external knowledge into the model. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods and fills the gap in multimodal document extraction. |
How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM (2025.coling-main)
Copied to clipboard
| Challenge: | Many large language models (LLMs) support many languages, while others only support a few, e.g. the Llama series. |
| Approach: | They present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. |
| Outcome: | The proposed model can be used to perform multilingual tasks on 1 to 52 languages. |
An Invariant Learning Characterization of Controlled Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Controlled generation is a problem of creating text that contains stylistic or semantic attributes of interest. |
| Approach: | They propose a distribution shift-based control system that can be used to train a predictor of the desired attribute. |
| Outcome: | The proposed method shows that the most effective predictor should be invariant across multiple text environments. |
Leveraging Explicit Procedural Instructions for Data-Efficient Action Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | a paper presents a data-efficient solution to constructing task-oriented dialogue systems . large language models have shown success in modeling such dialogues, but they require large quantities of data . |
| Approach: | They propose a system that leverages explicit instructions from agent guidelines . they propose dialogue-document matching and action-oriented masked language modeling . |
| Outcome: | The proposed system improves accuracy predicting in- and out-of-distribution actions while preserving high performance in settings with low or sparse data. |
A Survey on Zero Pronoun Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Zero pronouns (ZPs) are often omitted in pro-drop languages, but should be recalled in non-pro-drop language. |
| Approach: | They propose to analyze the literature on zero pronoun translation after the neural revolution . they uncover that data limitation causes learning bias in languages and domains . |
| Outcome: | The proposed method and methods are compared to other models and evaluation metrics on different benchmarks. |
Interpretable Graph-Language Modeling for Detecting Youth Illicit Drug Use (2026.findings-eacl)
Copied to clipboard
Yiyang Li, Zehong Wang, Zhengqing Yuan, Zheyuan Zhang, Keerthiram Murugesan, Chuxu Zhang, Yanfang Ye
| Challenge: | Illicit drug use among teens and young adults remains a public health concern . existing models ignore latent and interconnected structures among survey variables . |
| Approach: | They propose a joint graph-language modeling framework to detect illicit drug use among TYAs . they use large-scale surveys such as the Youth Risk Behavior Survey and the National Survey on Drug Use and Health to analyze data . |
| Outcome: | The proposed framework outperforms baseline models on YRBS and NSDUH datasets in predictive accuracy. |
Word Alignment as Preference for Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Hallucination and omission are a problem in machine translation because of an LLM's size and low-resource languages. |
| Approach: | They propose to use word alignment as preference to optimize an LLM-based MT model to mitigate hallucination and omission problems. |
| Outcome: | The proposed model is able to mitigate hallucination and omission by using word alignment as preference. |
Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparison (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to extract examples from memory are limited, but the upstream retrieval step is still unexplored. |
| Approach: | They propose to use a standard autoregressive model, edit-based model and a large language model with in-context learning to investigate the effect of retrieval methods on translation scores. |
| Outcome: | The proposed architectures improve translation scores and increase diversity of examples. |
Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate multiple independent CoTs, combining them through ensembling or other post-hoc strategies, have been shown to be effective in boosting performance. |
| Approach: | They propose a method where LLMs are fine-tuned to generate a sequence of Diverse Chains of Thought (DCoT) within a single inference step. |
| Outcome: | The proposed model can generate multiple chains of thought within a single inference step without external feedback. |
Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models (2024.findings-naacl)
Copied to clipboard
Wanyong Feng, Jaewook Lee, Hunter McNichols, Alexander Scarlatos, Digory Smith, Simon Woodhead, Nancy Ornelas, Andrew Lan
| Challenge: | Multiple-choice questions (MCQs) are easy to administer and grade . but crafting high-quality distractors remains labor-intensive and limited scalability . |
| Approach: | They propose to automate the generation of distractors in math MCQs by using large language models to generate distractors. |
| Outcome: | The proposed methods can generate valid distractors, but they are less adept at anticipating common errors or misconceptions among real students. |
Making Language Models Better Tool Learners with Execution Feedback (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing tool learning methodologies induce large language models to utilize tools indiscriminately . Existing frameworks that teach language models when and how to use tools propagate errors rather than enhance performance. |
| Approach: | They propose a framework that enables large language models to continually learn through feedback derived from tool execution. |
| Outcome: | The proposed framework can make large language models selectively use tools . it improves accuracy while enhancing insufficient tool learning, it shows . |
Can LLMs Facilitate Interpretation of Pre-trained Language Models? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to uncover knowledge encoded within pre-trained language models are limited in terms of scalability and scope of interpretation. |
| Approach: | They propose to use a large language model, ChatGPT, as an annotation tool . they demonstrate that ChatGPt produces accurate and semantically richer annotations . |
| Outcome: | The proposed method produces accurate and semantically richer annotations compared to human annotations. |
Human-in-the-Loop Synthetic Text Data Inspection with Provenance Tracking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Data augmentation techniques generate low-quality texts with incorrect labels . a new technique is needed to winnow out texts with inaccurate labels based on provenance inspection . |
| Approach: | They develop a data inspection technique that uses provenance inspection and assistive labeling to winnow out texts with incorrect labels. |
| Outcome: | a new human-in-the-loop data inspection technique can winnow out texts with incorrect labels . the technique can reduce human inspection effort by combining provenance inspection and assistive labeling . |
REFLEX: Self-Refining Explainable Fact-Checking via Verdict-Anchored Style Control (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for automated fact-checking often overlook deceptive misinformation styles in generated explanations. |
| Approach: | They propose a framework that explicitly controls reasoning style by anchoring explanations to the predicted verdict. |
| Outcome: | The proposed framework achieves state-of-the-art under LLaMA-series models with 465 samples. |
From Curiosity to Clarity : Exploring the Impact of Consecutive Why-Questions (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study has demonstrated the utility of consecutive why-questions in everyday life. |
| Approach: | They used a WHY-Chain dataset to construct a model that asked a why-questions question . they also used objectives that capture the 'consecutive' characteristic of the data . |
| Outcome: | The proposed model performed better on downstream tasks that require commonsense reasoning . the model was validated by ablation studies and the validity check . |
Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing long-text evaluation benchmarks, such as L-Eval and LongBench, focus on QA and summarization tasks. |
| Approach: | They propose a length-adaptable benchmark for evaluating the long-context understanding of large language models. |
| Outcome: | The proposed benchmarks do not cover ultralong settings (100k+ tokens) and are difficult to evaluate across different length ranges. |
Molecular Facts: Desiderata for Decontextualization in LLM Fact Verification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used to combat hallucinations . granularity of fact-checking makes it difficult to fact- check larger chunks of text . |
| Approach: | They propose a method for generating molecular facts automatically using decontextuality and minimality. |
| Outcome: | The proposed method balances minimality with fact verification accuracy in ambiguous settings. |
CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on counterfactual data augmentation only considered restricted classes of perturbations, limiting their effectiveness. |
| Approach: | They propose a retrieval-augmented framework for creating diverse counterfactual perturbations for CDA. |
| Outcome: | Experiments on natural language inference and sentiment analysis show that the proposed framework can be used to encourage diversity in manually authored perturbations. |
Plug-and-Play Conversational Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large conversational models that generate coherent and fluent responses often require large dialogue datasets. |
| Approach: | They propose and evaluate plug-and-play methods for controllable response generation . they demonstrate a high degree of control over the generated conversational responses . |
| Outcome: | The proposed method does not require further computation at decoding time and does not need fine-tuning of a large language model. |
Retrieval, Reasoning, Re-ranking: A Context-Enriched Framework for Knowledge Graph Completion (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing embedding-based methods rely on triples in the KG, which is vulnerable to specious relation patterns and long-tail entities. |
| Approach: | They propose a context-enriched framework for KGC that uses a large language model to generate potential answers for each query triple. |
| Outcome: | The proposed framework improves on FB15k237 and WN18RR datasets. |
BP4ER: Bootstrap Prompting for Explicit Reasoning in Medical Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Medical dialogue generation (MDG) has gained increasing attention due to its substantial practical value. |
| Approach: | They propose a method which explicitly models MDG’s multi-step reasoning process and iteratively enhances this reasoning process. |
| Outcome: | The proposed method outperforms state-of-the-art methods across objective and subjective evaluations on two publicly available datasets. |
AlignCap: Aligning Speech Emotion Captioning to Human Preferences (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for speech emotion capture often produce hallucinations and lose generalization on unseen speech. |
| Approach: | They propose to align speech emotion captioning to human preference based on large language model (LLM) and human preference regularization to eliminate factuality and faithfulness hallucinations. |
| Outcome: | Experiments show that AlignCap performs better than existing methods on Zero-shot SEC task. |
PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. |
| Approach: | They propose a method to optimize prompts for LLM-driven multi-step tasks using a human-designed feedback rule. |
| Outcome: | The proposed method outperforms human-engineered prompts and several other prompt optimization methods on 11 representative multi-step tasks. |
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing VideoQA models struggle to adapt to new questions or tasks posed by newly available content. |
| Approach: | They propose a continual learning framework that fine-tunes a large language model for a sequence of tasks and integrates specific question constraint prompting, knowledge acquisition prompting and visual temporal awareness prompting. |
| Outcome: | The proposed model achieves 55.14% accuracy on both NExT-QA and DramaQA datasets and 71.24% accuracy for DramaQA. |
CycleOIE: A Low-Resource Training Framework For Open Information Extraction (2025.coling-main)
Copied to clipboard
Zhihong Jin, Chunhong Zhang, Zheng Hu, Jibin Yu, Ruiqi Ma, Qingyun Chen, Xiaohao Liao, Yanxing Zhang
| Challenge: | Open Information Extraction (OpenIE) models rely heavily on large amounts of annotated data. |
| Approach: | They propose a training framework that maximizes data efficiency through a cycle-consistency mechanism. |
| Outcome: | The proposed approach improves the quality of training data by curating low-quality datasets annotated by a large language model. |
Automatic Table Union Search with Tabular Representation Learning (2023.findings-acl)
Copied to clipboard
Xuming Hu, Shen Wang, Xiao Qin, Chuan Lei, Zhengyuan Shen, Christos Faloutsos, Asterios Katsifodimos, George Karypis, Lijie Wen, Philip S. Yu
| Challenge: | Existing methods to identify uniability based on column representations are insufficient to reveal latent relational features to describe column relation between pair of columns. |
| Approach: | They propose a self-supervised table union search framework called AutoTUS to learn column relational representations in a multi-stage manner. |
| Outcome: | The proposed framework improves on the SOTA baseline and on real-world datasets. |
FeRG-LLM : Feature Engineering by Reason Generation Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | FeRG-LLM is a large language model that performs feature engineering at an 8billion-parameter scale. |
| Approach: | They propose a framework to perform feature engineering at an 8billion-parameter scale using conversational dialogues. |
| Outcome: | The proposed framework outperforms Llama 3.1 70B and Llma 3.2 on most datasets while using fewer resources and achieving reduced inference time. |
Emosical: An Emotion-Annotated Musical Theatre Dataset (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Emosical provides rich emotion annotations for musical films by inferring the background story of the characters. |
| Approach: | They propose to use a multimodal dataset of musical films to generate annotated emotion tags for each sample by inferring the background story of the characters. |
| Outcome: | The proposed dataset provides rich emotion annotations for musical films by inferring the background story of the characters. |
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on logical reasoning tasks; it remains unclear whether improvements hold for more diverse types of reasoning, especially in socially situated contexts. |
| Approach: | They perform a controlled evaluation of zero-shot CoT reasoning in two socially sensitive domains: harmful questions and stereotype benchmarks. |
| Outcome: | The results show that zero-shot CoT reasoning increases model’s likelihood to produce harmful or undesirable output, but decreases with improved instruction following. |
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering (2024.emnlp-main)
Copied to clipboard
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli
| Challenge: | Existing datasets for question answering based on retrieval augmented generation (RAG-QA) are either constructed using a single source corpus or consist of short extractive answers, which fall short of evaluating large language model (LLM) based RAG-QA systems on cross-domain generalization. |
| Approach: | They propose a dataset that integrates short extractive answers from multiple documents into a single coherent narrative. |
| Outcome: | The proposed dataset integrates short extractive answers from multiple documents into a single coherent narrative, covering 26K queries and large corpora across seven different domains. |
CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in artificial intelligence highlight the potential of language models in psychological health support. |
| Approach: | They propose a method to enhance the precision and efficacy of psychological support through large language models. |
| Outcome: | The proposed model generates professional and structured responses in Chinese psychological health Q&A tasks, showcasing its practicality and quality. |
Chat-TS: Enhancing Multi-Modal Reasoning Over Time-Series and Natural Language Data (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models are being rapidly applied across many fields such as healthcare, finance, transportation, and energy. |
| Approach: | They propose a large language model framework that integrates time-series tokens into LLMs’ vocabulary, enhancing its reasoning ability over time- and textual data. |
| Outcome: | The proposed framework enhances reasoning ability over time-series and textual data without compromising core natural language capabilities. |
Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite advances in improving large language model (LLM) to refuse to answer malicious instructions, LLMs remain vulnerable to jailbreak attacks where attackers generate instructions with distributions differing from safety alignment corpora. |
| Approach: | They propose a framework that leverages embedding space distribution analysis to generate jailbreak-like instructions. |
| Outcome: | The proposed framework shows significant decreases in attack success rate on Qwen2.5, Llama3.1, and Llma3.2 without compromising their utility. |
InstructEval: Systematic Evaluation of Instruction Selection Methods (2024.findings-naacl)
Copied to clipboard
| Challenge: | In-context learning (ICL) performs tasks by prompting a large language model using an instruction and a small set of annotated examples. |
| Approach: | They develop an ICL evaluation suite to evaluate the performance of popular instruction selection methods. |
| Outcome: | The proposed evaluation suite compares instruction selection methods over five metrics relevant to ICL. |
HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-depth research on the specific capabilities needed by the RAG generation model is lacking, leading to inconsistent document quality and retrieval system imperfections. |
| Approach: | They propose that RAG models should possess three progressively hierarchical abilities: (1) Filtering: the ability to select relevant information; (2) Combination: the capability to combine semantic information across paragraphs; (3) RAG-specific reasoning: the capacity to further process external knowledge using internal knowledge. |
| Outcome: | Experiments show that the proposed method significantly improves the model’s open-book examination capability on datasets such as RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA. |
Diagnosis of Dysarthria Severity and Explanation Generation Using XAI-Enhanced CLINIC-GENIE on Diadochokinetic Tasks (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent deep learning approaches for dysarthria impairment severity lack interpretability essential for clinical applications. |
| Approach: | They propose a deep neural network classifier that integrates acoustic and speech embeddings with Clinically Explainable Acoustic Features (CEAFs) and a module that transforms CEAFs and their Shapley values into intuitive natural language explanations. |
| Outcome: | The proposed model achieves a balanced accuracy of 0.952 (17.3% improvement over using CEAFs alone) and certified speech-language pathologists rated explanations with an average fidelity score of 4.94, confirming enhanced clinical utility. |
Know your audience: specializing grounded language models with listener subtraction (2023.eacl-main)
Copied to clipboard
| Challenge: | Effective communication requires adapting to the idiosyncrasies of each communicative context. |
| Approach: | They propose a method for specializing grounded language models without supervision . they fine-tune an attention-based adapter between a CLIP vision encoder and a large language model . |
| Outcome: | The proposed method allows a speaker to adapt to the idiosyncracies of the listeners without supervision. |
Anticipating Future with Large Language Model for Simultaneous Machine Translation (2025.naacl-long)
Copied to clipboard
Siqi Ouyang, Oleksii Hrinchuk, Zhehuai Chen, Vitaly Lavrukhin, Jagadeesh Balam, Lei Li, Boris Ginsburg
| Challenge: | Existing methods only use the partial utterance that has already arrived at the input and the generated hypothesis. |
| Approach: | They propose to use a large language model to predict future source words and opportunistically translate without introducing too much risk. |
| Outcome: | The proposed method outperforms baselines on four language directions and achieves the best translation quality-latency trade-off by up to 5 BLEU points at the same latency. |
FS-Researcher: Test-Time Scaling for Long-Horizon Research Tasks with File-System-Based Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Long trajectories in deep research often exceed model context limits, compressing token budgets for both evidence collection and report writing. |
| Approach: | They propose a file-system-based framework that scales deep research beyond context window . a Context Builder agent acts as a librarian and a Report Writer agent composes the final report . |
| Outcome: | Experiments on two open-ended benchmarks show that FS-Researcher achieves state-of-the-art report quality across different backbone models. |
CLEVR-POC: Reasoning-Intensive Visual Question Answering in Partially Observable Environments (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing knowledge for reasoning about partially observed scenes is limited . lucian et al. show that pre-trained vision language models are not adequate for VQA . |
| Approach: | They propose a benchmark for reasoning-intensive visual question answering . they use logical constraints to leverage knowledge to generate plausible answers . |
| Outcome: | The proposed model performs better than pre-trained models on CLEVR-POC . the proposed model is based on a neuro-symbolic model with a visual perception network and a formal logical reasoner . |
Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks World (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on how large language model agents collaborate with humans in equal roles emphasize the importance of coordination and communication. |
| Approach: | They propose to use chain-of-thought prompts to evaluate different collaboration perspectives, from independent to more complex, dependent tasks. |
| Outcome: | The proposed model significantly improves the evaluation metric. |
LLatrieval: LLM-Verified Retrieval for Verifiable Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models struggle with factual errors and often produce non-factual and fabricated content. |
| Approach: | They propose to use large language models to generate text with supporting documents to enable the user to flexibly verify the answer. |
| Outcome: | Experiments on ALCE show that LLatrieval significantly outperforms extensive baselines and achieves state-of-the-art results. |
How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a systematic and comprehensive empirical evaluation of state-of-the-art reranking methods is presented. |
| Approach: | They evaluate 22 reranking methods including 40 variants across established benchmarks . primary goal is to determine whether performance disparity exists between LLM-based reranters and lightweight counterparts based on novel queries . |
| Outcome: | The proposed methods perform better on familiar queries than lightweight models, the authors show . |
Automatic Evaluation of Attribution by Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Generative large language models (LLMs) incorporate external references to generate and support claims. however, evaluating the attribution remains an open problem. |
| Approach: | They investigate automatic evaluation of attribution given by large language models . they define different types of attributed errors and then explore two approaches . |
| Outcome: | The proposed methods highlight promising signals and challenges. |
NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering (2025.acl-long)
Copied to clipboard
Ruisheng Cao, Hanchong Zhang, Tiancheng Huang, Zhangyi Kang, Yuxin Zhang, Liangtai Sun, Hanqi Li, Yuxun Miao, Shuai Fan, Lu Chen, Kai Yu
| Challenge: | Existing approaches to retrieval augmented generation neglect PDF structure and layout . individual PDFs often exceed prompt limits and user queries may span multiple documents. |
| Approach: | They propose a hybrid neural symbolic retrieval framework which combines both paradigms in an interactive process. |
| Outcome: | The proposed framework organizes semi-structured PDF content into relational database and vectorstore . it defeats both RAG and structured baselines on three PDF-based QA datasets . |
Controllable Dialogue Simulation with In-context Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to generate annotated dialogues require crowdsourcing, which is expensive and time-consuming. |
| Approach: | They propose a dialogue simulation method based on large language model in-context learning that generates new dialogues and annotations in a controllable way. |
| Outcome: | The proposed method can expand a small set of dialogue data with minimum or zero human involvement and parameter update. |
Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing end-to-end task-oriented dialogue systems require extensive training datasets to perform well. |
| Approach: | They propose a system that synergizes LLMs with task-specific hints to improve alignment in low-data settings. |
| Outcome: | The proposed model improves alignment in low-data settings while retaining competitive performance in full-data environments. |
Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are robust and general, but their reasoning ability is not at a level to compete with the best models trained for specific natural language reasoning problems. |
| Approach: | They propose to use large language models as a few-shot semantic parser to convert natural language sentences into a logical form that serves as input for answer set programs. |
| Outcome: | The proposed model can handle multiple question-answering tasks without requiring retraining for each new task. |
LARA: LLM-based Agile Power Distribution Network Restoration from Disastrous Events (2026.findings-eacl)
Copied to clipboard
| Challenge: | a large language model generates high-level restoration plans over a compact catalogue of feasible actions. |
| Approach: | They propose a method that generates restoration plans over a catalogue of feasible actions. |
| Outcome: | The proposed model outperforms a time-capped solver on an IEEE 13-node power distribution feeder by 13% while using less than 1% of its wall-clock runtime. |
SILC-EFSA: Self-aware In-context Learning Correction for Entity-level Financial Sentiment Analysis (2025.coling-main)
Copied to clipboard
Senbin Zhu, ChenYuan He, Hongde Liu, Pengcheng Dong, Hanjie Zhao, Yuchen Yan, Yuxiang Jia, Hongying Zan, Min Peng
| Challenge: | Currently, most sentiment analysis corpora use sequence-level annotation. |
| Approach: | They propose a two-stage approach to financial entity-level sentiment analysis called Self-aware In-context Learning Correction. |
| Outcome: | The proposed approach achieves state-of-the-art on the largest English and Chinese financial entity-level sentiment analysis datasets to date. |
XMC-Agent : Dynamic Navigation over Scalable Hierarchical Index for Incremental Extreme Multi-label Classification (2024.findings-acl)
Copied to clipboard
Yanjiang Liu, Tianyun Zhong, Yaojie Lu, Hongyu Lin, Ben He, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun
| Challenge: | Existing methods for XMC struggle with the growing set of labels due to their static label assumptions, and embedding-based methods struggle with complex mapping relationships due to late interaction paradigm. |
| Approach: | They propose a large language model (LLM) powered agent framework for extreme multi-label classification, XMC-Agent, which can effectively learn, manage and predict the extremely large and dynamically increasing set of labels. |
| Outcome: | The proposed framework can learn, manage and predict the extremely large and dynamically growing set of labels and achieves state-of-the-art performance on three standard datasets. |
Logit Separability-Driven Samples and Multiple Class-Related Words Selection for Advancing In-Context Learning (2025.naacl-long)
Copied to clipboard
| Challenge: | Effective organization of in-context learning (ICL) demonstrations is key to improving the quality of large language models (LLMs). |
| Approach: | They propose a logit separability-based method that integrates multiple class-related words into each sample-label pair to improve LLM understanding. |
| Outcome: | The proposed method improves ICL performance by providing clearer instructions and richer label information. |
Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models lack commonsense knowledge . |
| Approach: | They propose a contextualized knowledge prompting scheme to guide the contextualization of structured commonsense knowledge based on large language models. |
| Outcome: | The proposed approach outperforms the state-of-the-art technique by an average of 5.8%. |
From Distributional to Overton Pluralism: Investigating Large Language Model Alignment (2025.naacl-long)
Copied to clipboard
| Challenge: | a large language model's (LLM) output distribution is changed by an alignment process . a recent study shows that aligned models surface information that cannot be recovered from base models without fine-tuning. |
| Approach: | They analyze two aspects of the alignment process that change output distributions . they find alignment suppresses irrelevant and unhelpful content . |
| Outcome: | The proposed model can be imitated without fine-tuning by using in-context examples and lower-resolution semantic hints about response content. |
Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Yeonjun In, Wonjoong Kim, Kanghoon Yoon, Sungchul Kim, Mehrab Tanjim, Sangwu Park, Kibum Kim, Chanyoung Park
| Challenge: | Extensive benchmarks evaluate LLM safety relying heavily on general standards . no benchmark datasets exist to evaluate the user-specific safety of LLMs . |
| Approach: | a new benchmark is designed to assess user-specific aspect of LLM safety . authors propose a simple remedy based on chain-of-thought to improve user-specified safety. |
| Outcome: | a new benchmark assesses the user-specific aspect of LLM safety . the proposed solution improves user-specified safety by chain-of-thought . |
FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking (2024.findings-emnlp)
Copied to clipboard
Zhuoer Wang, Leonardo Ribeiro, Alexandros Papangelis, Rohan Mukherjee, Tzu-Yen Wang, Xinyan Zhao, Arijit Biswas, James Caverlee, Angeliki Metallinou
| Challenge: | Existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user’s request. |
| Approach: | They propose an output-side optimization approach that uses state-tracked constrained decoding and a lightweight model as the discriminator to rerank the beam-searched candidate generations of the large language model. |
| Outcome: | The proposed approach outperforms existing supervised and in-context learning approaches in the DSTC8 and API Bank datasets in terms of call generation accuracy, inference efficiency, and context efficiency. |
GeoAgent: To Empower LLMs using Geospatial Tools for Address Standardization (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to address address standardization are lacking in the current field. |
| Approach: | They propose a framework that incorporates spatial knowledge into address texts and achieves efficient address standardization. |
| Outcome: | The proposed framework incorporates spatial knowledge into address texts and achieves efficient address standardization. |
Llamipa: An Incremental Discourse Parser (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Discourse parsing is a task of predicting relationships between utterances and their semantic content . lack of surface cues in discourse graphs forces parsers to rely on deep, semantic information . a large language model (LLM) can significantly improve discourse parser performance . |
| Approach: | They propose a large language model (LLM) that leverages discourse context to parse a discourse . this model provides local, context-sensitive representations of discourse units . |
| Outcome: | The proposed model can provide local, context-sensitive representations of discourse units . it can process discourse data incrementally, which is essential for later use of discourse information . |
Continual Learning Using Only Large Language Model Prompting (2025.coling-main)
Copied to clipboard
| Challenge: | Existing continuous learning paradigms fine-tune language model parameters or use adapters or variants to adapt the LM. |
| Approach: | They propose a new continual learning paradigm wherein a large language model is regarded as a black box. |
| Outcome: | The proposed method outperforms baselines by a large margin in learning tasks incrementally. |
Inference Scaling for Bridging Retrieval and Augmented Generation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing work observed the generator bias, such that improving the retrieval results may negatively affect the outcome. |
| Approach: | They propose to use inference scaling to aggregate inference calls from the permuted order of retrieved contexts to create a new ranking. |
| Outcome: | The proposed approach improves ROUGE-L on MS MARCO and EM on HotpotQA benchmarks by 7 points. |
Detecting Conceptual Abstraction in LLMs (2024.lrec-main)
Copied to clipboard
| Challenge: | a novel approach to detecting noun abstraction within a large language model is proposed . a first step towards the explainability of conceptual abstraction in LLMs is shown . |
| Approach: | They propose a method to detect noun abstraction within a large language model . they instantiate taxonomic relationships and analyze attention matrices produced by BERT . |
| Outcome: | The proposed approach can detect hypernymy in a large language model . the results are a first step towards the explainability of conceptual abstraction in LLMs . |
Embracing Large Language Models in Traffic Flow Forecasting (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict future traffic flows capture spatio-temporal dependencies, but they fail to adapt to test-time environmental changes. |
| Approach: | They propose to use large language models to help traffic flow forecasting by capturing spatio-temporal dependencies and using a large language model to select the most likely result. |
| Outcome: | The proposed method is based on large language models (LLMs) and an LLM-based selector. |
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval (2024.naacl-long)
Copied to clipboard
| Challenge: | et al., 2020: performance of dense retrieval models in multilingual retrieval is limited due to uneven and scarce training data available across multiple languages. |
| Approach: | They propose a synthetic retrieval training dataset containing 33 languages for fine-tuning multilingual retrievers without human supervision. |
| Outcome: | The proposed model outperforms human-supervised retrieval models on three retrieval benchmarks. |
T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Text-to-image (T2I) generative models have demonstrated exceptional capability in synthesizing high-quality images from textual prompts. |
| Approach: | They propose a benchmark to explore the knowledge-driven reasoning capabilities of T2I models. |
| Outcome: | The proposed benchmark examines the knowledge-driven reasoning capabilities of T2I models. |
Prompt-Based Monte-Carlo Tree Search for Goal-oriented Dialogue Policy Planning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Optimal policy planning is a difficult task, authors say . many goal-oriented conversations require subjective strategies, they say - a problem in goal-orientated settings . |
| Approach: | They propose an approach to perform goal-oriented dialogue policy planning without model training. |
| Outcome: | The proposed approach performs goal-oriented dialogue policy planning without model training. |
Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents via Semantic-Oriented Hierarchical Graphs (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing work on document visual question answering fails to capture the differences and correlations between elements of a document and associated questions. |
| Approach: | They propose a document-visual question-answering challenge that exploits element-level semantics and employs hierarchical Graph structures to capture differences and correlations between elements. |
| Outcome: | The proposed model surpasses the state-of-the-art method and large language model in terms of Exact Match (EM) metric, demonstrating exceptional effectiveness. |
Document-Level Event Extraction via Information Interaction Based on Event Relation and Argument Correlation (2024.lrec-main)
Copied to clipboard
| Challenge: | Document-level Event Extraction (DEE) is a vital task in NLP . current approaches overlook intricate relationships among events and subtle correlations among arguments within a document . |
| Approach: | They propose a document-level event extraction tool that integrates event relationships and argument correlation graphs to model the relationship among events. |
| Outcome: | The proposed network outperforms existing models and large language models in terms of F1-score across two benchmark datasets. |
mDPO: Conditional Preference Optimization for Multimodal Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have attempted to apply DPO to multimodal scenarios but have found it challenging to achieve consistent improvement. |
| Approach: | They propose a multimodal DPO objective that prevents the over-prioritization of language-only preferences by also optimizing image preference. |
| Outcome: | The proposed method significantly improves performance on two multimodal LLMs of different sizes and three widely used benchmarks. |
Language-to-Code Translation with a Single Labeled Example (2024.emnlp-main)
Copied to clipboard
Kaj Bostrom, Harsh Jhamtani, Hao Fang, Sam Thomson, Richard Shin, Patrick Xia, Benjamin Van Durme, Jason Eisner, Jacob Andreas
| Challenge: | In-Context Inverse Programming (ICIP) bootstraps a language-to-code system using mostly unlabeled programs written using a potentially unfamiliar library or API. |
| Approach: | They propose a method for bootstrapping a language-to-code system using mostly unlabeled programs written using a potentially unfamiliar library or API. |
| Outcome: | The proposed method outperforms baselines across nine domains and 100 examples in a “nearly unsupervised” setting. |
To Tell The Truth: Language of Deception and Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing evidence of people’s ability to discern truth from text-based false information is scarce. |
| Approach: | They propose to use a large language model to learn discernible cues from TV game show data to investigate whether textual cue is more likely to detect fraud . |
| Outcome: | The proposed model detects novel but accurate language cues in many cases where humans failed to detect deception. |
MisinfoEval: Generative AI in the Era of “Alternative Facts” (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing efforts to address misinformation on social media platforms are hampered by user biases and scalability challenges. |
| Approach: | They propose a framework for generating and comprehensively evaluating large language model based misinformation interventions using a simulated social media environment and personalized explanations tailored to users' beliefs. |
| Outcome: | The proposed framework improves accuracy at reliability labeling by up to 41.72% and personalized explanations appeal to users' pre-existing values. |
HARE: an entity and relation centric evaluation framework for histopathology reports (2025.findings-emnlp)
Copied to clipboard
| Challenge: | evaluating the clinical quality of medical domain automated text generation remains a challenge. |
| Approach: | They propose a framework for histopathology automated report evaluation that prioritizes clinically relevant content by aligning critical histo pathology entities and relations between reference and generated reports. |
| Outcome: | The proposed framework outperforms existing metrics in histopathology report evaluations. |
Boosting the Capabilities of Compact Models in Low-Data Contexts with Large Language Models and Retrieval-Augmented Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing language models lack data and computation power, but they are extremely parameter-heavy and difficult to train. |
| Approach: | They propose a retrieval augmented generation framework backed by a large language model to correct the output of a smaller model for morphological glossing. |
| Outcome: | The proposed model is highly effective in data-scarce settings and offers a state-of-the-art for morphological glossing. |
Improving LLM Generations via Fine-Grained Self-Endorsement (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable capabilities but can still fail frequently on knowledge-intensive tasks. |
| Approach: | They propose a self-endorsement framework that leverages fine-grained fact-level comparisons across multiple sampled responses. |
| Outcome: | The proposed framework can improve factuality of generations with simple prompts across scales of LLMs. |
EmpCRL: Controllable Empathetic Response Generation via In-Context Commonsense Reasoning and Reinforcement Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies lack the perception of fine-grained dialogue emotion propagation, and have limitations in reasoning about the intentions of users on cognition, which affect the quality of empathetic response. |
| Approach: | They propose to use commonsense reasoning and reinforcement learning to generate empathetic response based on in-context commonsensing and contextual reasoning to broaden cognitive boundaries. |
| Outcome: | The proposed model outperforms state-of-the-art models in automatic and human evaluation. |
Evaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awareness (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on editing LLMs neglects the dependency between a fact and its logical implications. |
| Approach: | They propose an evaluation protocol that assesses the editing process using StandUp . they aim to ensure that the editing respects internal logical constraints . |
| Outcome: | The proposed evaluation protocol assesses the editing process using a standup question-answering dataset. |
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling (2024.acl-long)
Copied to clipboard
Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yu-Gang Jiang, Xipeng Qiu
| Challenge: | Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output. |
| Approach: | They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities. |
| Outcome: | The proposed model can be trained stably without any alterations to existing models or training paradigms. |
APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs (2025.acl-long)
Copied to clipboard
Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Sun Ao, Hao Zhou, Jie Zhou, Zhiyuan Liu, Maosong Sun
| Challenge: | Long-context inference is crucial for advancing large language models, but its prefill speed remains a bottleneck. |
| Approach: | They propose an efficient long-context inference framework that leverages multi-host approximate attention to enhance prefill speed. |
| Outcome: | The proposed framework achieves speedups of 9.2, 4.2, and 1.6 without any degradation in performance. |
PPT: A Minor Language News Recommendation Model via Cross-Lingual Preference Pattern Transfer (2025.acl-long)
Copied to clipboard
| Challenge: | Existing systems for minor language news recommendation lack interaction with content . linguistic gap can lead to inaccurate modeling of minor language content despite strong English capability . |
| Approach: | They propose a minor language news recommendation model by cross-lingual preference pattern transfer . their model employs the widely used two-tower architecture and large language model as the backbone of the news encoder . |
| Outcome: | The proposed model outperforms existing models on 15 minor languages. |
Enhancing Knowledge Retrieval with Topic Modeling for Knowledge-Grounded Dialogue (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to knowledge retrieval are limited by the knowledge base encoder, but our work focuses on the knowledge-base encoder. |
| Approach: | They propose an approach that utilizes topic modeling on the knowledge base to improve retrieval accuracy and as a result, improve response generation. |
| Outcome: | The proposed approach can improve retrieval and generation performance on two datasets. |
EVIT: Event-Oriented Instruction Tuning for Event Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have made significant advances in event reasoning . however, smaller instruction-tuned models do not consistently demonstrate exceptional proficiency . |
| Approach: | They propose an event-oriented instruction tuning technique to train a large language model . they propose a structure named event quadruple which contains the structure and semantics of events . |
| Outcome: | The proposed model achieves competitive performances on event reasoning tasks. |
Precise Model Benchmarking with Only a Few Observations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Accurate evaluation of large language models is crucial for identifying their strengths and weaknesses. |
| Approach: | They propose an empirical Bayes estimator that balances direct and regression estimates for each subgroup separately, improving the precision of subgroup-level estimates of model performance. |
| Outcome: | The proposed model reduces the mean squared error by up to 50% on multiple datasets. |
Measuring Contextual Informativeness in Child-Directed Text (2025.coling-main)
Copied to clipboard
Maria R. Valentini, Téa Y. Wright, Ali Marashian, Jennifer M. Ellis, Eliana Colunga, Katharina von der Wense
| Challenge: | Recent advances in natural language processing (NLP) have made it possible to generate children's stories with a single word. |
| Approach: | They propose a task of measuring contextual informativeness in children's stories and a large language model to automate the task. |
| Outcome: | The proposed method outperforms baselines and can generalize to measuring contextual informativeness in adult-directed text. |
AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training (2025.emnlp-main)
Copied to clipboard
| Challenge: | Empirically, AdamS demonstrates strong performance in various tasks . et al., 2023b): AdamS is efficient, efficient, and model-agnostic. |
| Approach: | They propose a model-agnostic alternative to Adam for large language model pretraining and post-training. |
| Outcome: | The proposed method matches memory footprint of SGD with momentum while delivering superior performance. |
If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies assume that VLMs prioritize visual attributes to represent concepts. |
| Approach: | They propose a novel approach to characterize features important for VLMs using reinforcement learning. |
| Outcome: | The proposed approach characterizes features that are important for VLMs . it shows that spurious descriptions have a major role in VLM representations despite providing no helpful information. |
Question-Instructed Visual Descriptions for Zero-Shot Video Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models for video QA rely on complex architectures, expensive pipelines or closed models like GPTs. |
| Approach: | They propose a single instruction-aware open vision-language model to tackle videoQA using frame descriptions. |
| Outcome: | The proposed framework achieves higher performance than current state-of-the-art models on videoQA benchmarks. |
Enhancing Tool Retrieval with Iterative Feedback from Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods have shown that large language models can handle a certain amount of tools through in-context learning or fine-tuning. |
| Approach: | They propose to enhance tool retrieval with iterative feedback from the large language model by prompting the tool usage model to provide feedback for the tool retriever model in multi-round. |
| Outcome: | The proposed approach achieves advanced performance in both in-domain evaluation and out-of-domain assessment. |
Context Attribution with Multi-Armed Bandit Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to augmenting attribution with retrieval-augmented generation (RAG) focus on training models to explicitly cite context segments during generation, but their reliability remains unverifiable. |
| Approach: | They propose a framework that formulates context attribution as a combinatorial multi-armed bandit problem by using Linear Thompson Sampling to efficiently identify the most influential context segments while minimizing the number of model queries. |
| Outcome: | The proposed method reduces model queries by 30% while matching or exceeding the attribution quality of existing approaches. |
WildSci: Advancing Scientific Reasoning from In-the-Wild Literature (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language model reasoning focus on mathematics and coding domains, but scientific reasoning remains limited in other domains due to limited dataset coverage. |
| Approach: | They propose a framework for sustainable scientific reasoning QA generation by synthesizing a new dataset of domain-specific science questions from peer-reviewed literature. |
| Outcome: | The proposed framework and dataset enable scalable and sustainable research in scientific reasoning. |
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated that a large language model (LLM) can generate training data for another LLM, or for creating supplementary training materials, such as rationales. |
| Approach: | They conduct an in-depth investigation to understand why fine-tuning an LLM with responses generated by a LLM often yields better results than using responses generated from humans. |
| Outcome: | The proposed approach can be used to transfer knowledge from a larger model to a smaller one, or for creating supplementary training materials, such as rationales. |
KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph completion methods struggle with long-tail entities due to limited structural information and imbalanced distributions of entities. |
| Approach: | They propose a framework that integrates a large language model and a triple-based KGC retriever to alleviate the long-tail problem without incurring additional training overhead. |
| Outcome: | The proposed model reduces training overhead and finetuning costs on benchmark datasets. |
AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing contrastive methods that ignore the context of a large language model (LLM) fail to handle instances that vary in their amount of conflict, with static methods over-adjusting when conflict is absent. |
| Approach: | They propose a fine-grained, instance-level approach called AdaCAD which dynamically adjusts the degree of conflict based on the degree. |
| Outcome: | The proposed approach outperforms baselines and improves factuality of summaries by 6.19. |
EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing frameworks for leveraging background knowledge of narratives are limited. |
| Approach: | They propose a framework to ground free-texts to eventuality-centric KGs for narrative reasoning . their framework is based on a set of probabilistic probabilistic models that are grounded in the real world . |
| Outcome: | The proposed framework outperforms baseline models while providing interpretable evidence. |
You need to MIMIC to get FAME: Solving Meeting Transcript Scarcity with Multi-Agent Conversations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing tools for meeting summarization are limited due to privacy and expensive manual annotation. |
| Approach: | They propose a meeting synthesis framework that generates meeting transcripts on a given knowledge source by defining psychologically grounded participant profiles, outlining the conversation, and orchestrating a large language model (LLM) debate. |
| Outcome: | The proposed framework generates meeting transcripts on a given knowledge source by defining psychologically grounded participant profiles, outlining the conversation, and orchestrating a large language model debate. |
How Personality Traits Influence Negotiation Outcomes? A Simulation based on Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Psychological evidence reveals the influence of personality traits on decision-making. |
| Approach: | They propose a simulation framework centered on large language model agents with synthesized personality traits and propose empirical insights into the strategic impacts of Big Five personality traits on outcomes of bilateral negotiations. |
| Outcome: | The proposed model can reproduce behavioral patterns observed in human negotiations. |
Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies (2026.findings-acl)
Copied to clipboard
| Challenge: | Simultaneous machine translation requires high-quality translations under strict real-time constraints. |
| Approach: | They extend the action space of simultaneous machine translation with four adaptive actions . they adapt these actions in a large language model framework and construct training references . |
| Outcome: | The proposed framework improves semantic metrics and achieves lower delay compared to reference translations and salami-based baselines. |
Large Language Models Know What is Key Visual Entity: An LLM-assisted Multimodal Retrieval for VQA (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing visual language models struggle to capture longtail knowledge in the real world due to redundant visual information. |
| Approach: | They propose a method leveraging the reasoning capability of a large language model to identify key visual entities. |
| Outcome: | The proposed method outperforms other strong visual language model-based systems in two knowledge-intensive VQA benchmarks and performs comparably to models with 1-2 orders larger parameters. |
P3: Prompts Promote Prompting (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in prompt optimization have shown effectiveness of using multiple components to optimize models . however, such unilateral approaches often yield suboptimal results due to interdependent nature of these components. |
| Approach: | They propose a self-improvement framework that optimizes both system and user prompts . they use offline optimized prompts to promote online prompt optimization . |
| Outcome: | The proposed framework improves performance on general and reasoning tasks. |
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning (2024.acl-long)
Copied to clipboard
Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O’Mahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergun, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Chien, Sebastian Ruder, Surya Guthikonda, Emad Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, Sara Hooker
| Challenge: | Existing datasets in the English language are mostly in the realm of instruction fine-tuning . aya dataset, the Aya Collection, and the AYa Evaluation Suite are key resources . |
| Approach: | They aim to build a human-curated instruction-following dataset spanning 65 languages . they work with fluent speakers of languages from around the world to collect natural instances of instructions and completions . |
| Outcome: | The goal is to build a human-curated instruction-following dataset spanning 65 languages. |
How to Make LLMs Forget: On Reversing In-Context Knowledge Edits (2025.naacl-long)
Copied to clipboard
| Challenge: | In-context knowledge editing (IKE) is an efficient and efficient knowledge editing method (Zheng et al., 2022b; Gangadhar and Stratos, 2024) it can be misused to manipulate responses opaquely, e.g., insert misinformation or offensive content. |
| Approach: | They propose to detect and reverse IKE-edits using only the top-10 output probabilities of the next token, even in a black-box setting. |
| Outcome: | The proposed method can be detected with high accuracy even in a black-box setting, achieving over 80% accuracy in recovering original, unedited outputs across multiple LLMs. |
Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence (2022.emnlp-main)
Copied to clipboard
| Challenge: | researchers have posited Dungeons and Dragons as a challenge problem to test systems on various language-related capabilities. |
| Approach: | They frame Dungeons and Dragons specifically as a dialogue system challenge . they train a large language model to generate the next game turn, conditioning it on different information. |
| Outcome: | The proposed game generates the next conversational turn and predicts the state of the game given the dialogue history. |
When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Multi-agent debate (MAD) aims to improve large language model reasoning by letting multiple agents exchange answers and then aggregate their opinions. |
| Approach: | They propose a principled framework that joins sycophancy and self-bias to mitigate and quantify identity bias in multi-agent debate by removing identity markers from prompts. |
| Outcome: | The proposed framework joins identity-driven sycophancy and self-bias to mitigate and quantify identity bias in multi-agent debate. |
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale (2023.acl-long)
Copied to clipboard
| Challenge: | 70% of attention heads and 20% of the feed forward networks can be removed with minimal decline in task performance. |
| Approach: | They propose to investigate whether in-context learning is not uniform across all components of a large language model. |
| Outcome: | The proposed model can remove 70% of attention heads and 20% of feed forward networks with minimal decline in task performance. |
Definition Generation for Automatically Induced Semantic Frame (2024.findings-acl)
Copied to clipboard
| Challenge: | Semantic frames are conceptual structures that describe specific types of situations or events. |
| Approach: | They propose to generate frame definitions from a set of frame-evoking words using a large language model. |
| Outcome: | The proposed task incorporates frame element reasoning as chain-of-thought to enhance the inclusion of correct frame elements in the generated definitions. |
Both Matter: Enhancing the Emotional Intelligence of Large Language Models without Compromising the General Intelligence (2024.findings-acl)
Copied to clipboard
| Challenge: | Emotional Intelligence (EI) is a key concept in the field of human intelligence. |
| Approach: | They propose a method to enhance EI of large language models by naive fine-tuning on EI-related tasks. |
| Outcome: | The proposed method improves EI of two LLM-based assistants without compromising GI. |
Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompts can be challenging to design for arithmetic word problem solving. |
| Approach: | They propose to use training data to replace CoT with programs as the reasoning step . their results show that leveraging training data can improve generalization ability of prompts . |
| Outcome: | The proposed methods improve the generalization ability of prompts and the performance of fine-tuned smaller models in arithmetic word problem solving. |
We Are What We Repeatedly Do: Inducing and Deploying Habitual Schemas in Persona-Based Responses (2023.emnlp-main)
Copied to clipboard
| Challenge: | a variety of personas can be elicited from large language models, but they are opaque and unpredictable. |
| Approach: | They propose an approach to dialogue generation that retrieves relevant schemas to condition a large language model to generate persona-based responses. |
| Outcome: | The proposed method captures habitual knowledge and generates persona-based responses from a large language model. |
CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models (2024.findings-acl)
Copied to clipboard
Linhao Yu, Yongqi Leng, Yufei Huang, Shang Wu, Haixin Liu, Xinmeng Ji, Jiahui Zhao, Jinwang Song, Tingting Cui, Xiaoqing Cheng, Liutao Liutao, Deyi Xiong
| Challenge: | Recent years have witnessed remarkable progress achieved by large language models in both natural language understanding and generation. |
| Approach: | They propose a large benchmark CMoralEval for moral evaluation of Chinese LLMs . they use a Chinese TV program discussing Chinese moral norms and Chinese moral anomies based on various sources . |
| Outcome: | The proposed dataset is characterized by diversity and authenticity. |
Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training (2024.emnlp-main)
Copied to clipboard
Yixuan Wang, Xianzhen Luo, Fuxuan Wei, Yijun Liu, Qingfu Zhu, Xuanyu Zhang, Qing Yang, Dongliang Xu, Wanxiang Che
| Challenge: | Existing speculative decoding methods require additional model structure and training processes to assist the model for draft token generation. |
| Approach: | They propose a make some noise training framework that introduces some noise at the input for the model to learn the denoising task. |
| Outcome: | The proposed model improves inference speed by 2.3-2.7x times without compromising model performance. |
Prediction-Augmented Generation for Automatic Diagnosis Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) adopt autoregressive architecture, predicting the next word token based on the preceding context. |
| Approach: | They propose a method that integrates task-specific predictive models as external tools to improve model generation quality and accuracy. |
| Outcome: | The proposed method improves the generation quality and predictive accuracy of large language models in inference-driven tasks. |
DaNet: Dual-Aware Enhanced Alignment Network for Multimodal Aspect-Based Sentiment Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods assume a direct alignment between images and aspects, matching the entire image with a corresponding aspect. Existing algorithms assume 'direct alignment' between images, introducing noise. |
| Approach: | They propose a Dual-Aware Enhanced Alignment Network (DaNet) that can enhance fine-grained multimodal aspect-image alignment and denoising. |
| Outcome: | The proposed system outperforms existing methods in three subtasks and is available on https://github.com/***/DaNet. |
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts (2024.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for the automated evaluation of natural language texts are based on a large language model (LLM) that fails to agree with human judges and is not fully validated by the human judges. |
| Approach: | They propose a large language model (LLM) that generates a distribution over potential responses to assess multiple dimensions of interest. |
| Outcome: | The proposed framework predicts human judges' assessment of user satisfaction on a scale of 1–4 with an RMS error 0.5, a 2 improvement over the uncalibrated baseline. |
CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing vision-and-language navigation methods do not incorporate environmental feedback into their decision-making processes. |
| Approach: | They propose a framework that incorporates environmental feedback into decision-making and a 3D simulator that renders realistic scenarios using Unreal Engine 5. |
| Outcome: | The proposed framework outperforms existing vision-and-language navigation methods in a zero-shot multi-task setting by 28.1% on average. |
Better Process Supervision with Bi-directional Rewarding Signals (2025.findings-acl)
Copied to clipboard
Wenxiang Chen, Wei He, Zhiheng Xi, Honglin Guo, Boyang Hong, Jiazheng Zhang, Nijun Li, Tao Gui, Yun Li, Qi Zhang, Xuanjing Huang
| Challenge: | Existing processes that reward for each step are one-directional and lack a mechanism to model the distance to the final target. |
| Approach: | They propose a process supervision model that evaluates the correctness of previous steps and the probability of future success. |
| Outcome: | The proposed model outperforms existing supervision models like ORM and PRM on reasoning tasks and improves solution re-design. |
AlignUSER: Human-Aligned LLM Agents via World Models for Recommender System Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation practices for recommender systems rely on few-shot prompting and offline metrics are often misaligned with online behavior. |
| Approach: | They propose a framework that learns world-model-driven agents from human interactions. |
| Outcome: | The proposed framework enables agents to express rich preferences and feedback in natural language and interact with recommender systems in a simulation. |
On Collaborating Small and Large Models For Few-shot Intent Detection (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for few-shot intent detection face high inference cost and label interference. |
| Approach: | They propose a framework that integrates a small prediction model with a large language model for FSID. |
| Outcome: | The proposed framework outperforms existing methods on three benchmark datasets. |
Confabulation: The Surprising Value of Large Language Model Hallucinations (2024.acl-long)
Copied to clipboard
| Challenge: | 'confabulations' are inherently problematic and AI research should eliminate this flaw, but confabulation is not a problem. |
| Approach: | They argue that measurable semantic characteristics of large language model (LLM) hallucinations mirror a human propensity to utilize increased narrativity as a cognitive resource for sense-making and communication. |
| Outcome: | The proposed study shows that measurable semantic characteristics of LLM confabulations mirror human propensity to utilize increased narrativity as a cognitive resource for sense-making and communication. |
EconBERTa: Towards Robust Extraction of Named Entities in Economics (2023.findings-emnlp)
Copied to clipboard
Karim Lasri, Pedro Vitor Quinta de Castro, Mona Schirmer, Luis Eduardo San Martin, Linxi Wang, Tomáš Dulka, Haaya Naushan, John Pougué-Biyong, Arianna Legovini, Samuel Fraiberger
| Challenge: | Adapting general-purpose language models to specific domains has proven to be effective in tackling downstream tasks such as impact evaluation. |
| Approach: | They propose to use EconBERTa, a large language model pretrained on scientific publications in economics, and ECON-IE, based on an expert-annotated dataset of economics abstracts for Named Entity Recognition (NER). |
| Outcome: | The proposed model outperforms EconBERTa on the downstream NER task and ECON-IE on the economics abstracts. |
CentaurTA: A Self-Improving Human-Agents Collaboration Framework for Thematic Analysis (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing large language model approaches for qualitative analysis are labor-intensive and costly. |
| Approach: | They propose an iterative human–agent framework for scalable thematic analysis that integrates structured human feedback with rubric-based evaluation. |
| Outcome: | The proposed framework improves coding alignment and transparency across multiple datasets, baselines, and LLM families. |
Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognition (2024.findings-acl)
Copied to clipboard
| Challenge: | Mainstream of automatic speech recognition (ASR) has shifted from pipeline methods to end-to-end (E2E) methods. |
| Approach: | They propose to integrate a pre-trained speech representation model and a large language model (LLM) for automatic speech recognition in an end-to-end manner. |
| Outcome: | The proposed model achieves comparable performance to modern E2E ASR models by utilizing powerful pre-training models with the proposed integrated approach. |
Let’s Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | *Data Synthesis* is a promising way to train a small model with very little labeled data. |
| Approach: | They propose a framework that iteratively extrapolates the errors of a small model trained on a real-world validation dataset using a large language model. |
| Outcome: | The proposed framework reduces the gap between the synthesized dataset and the real data . it improves on multiple NLP tasks and on large models with human-annotated data. |
ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to shorten CoTs use length penalties or global entropy reduction . Existing approaches to CoT reasoning have significant practical drawbacks . |
| Approach: | They propose a method that shortens CoTs by length penalties or global entropy reduction . they integrate ETR into Group Relative Policy Optimization and evaluate it . |
| Outcome: | The proposed objective improves accuracy–efficiency trade-off by +9.9% while reducing CoT length by 67% across four benchmarks. |
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization (2023.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi
| Challenge: | a dataset of 1.5 million conversations distilled from everyday spoken situations is limited in scale due to its associated costs. |
| Approach: | They propose to make SODA a publicly available, million-scale high-quality social dialogue dataset . they contextualize social commonsense knowledge from a knowledge graph to distill broad spectrum of social interactions . |
| Outcome: | The proposed dataset is the first publicly available, million-scale high-quality social dialogue dataset. |
Natural Language Decompositions of Implicit Content Enable Better Text Representations (2023.emnlp-main)
Copied to clipboard
| Challenge: | a new method for the analysis of text takes implicitly communicated content into account . authors: human interpretation of each individual utterance is intractable . |
| Approach: | They propose a method that takes explicit communication into account when people interpret text . they use a large language model to generate propositions that are inferentially related to the text analyzed . |
| Outcome: | The proposed method proves useful in multiple problems that involve interpretation of utterances . it uncovers high-level narratives in public commentary, which are often not expressed in surface forms . |
Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks treat memory as a static append-only archive . Existing systems focus on passive accumulation, resulting in a 'passive accumulation' of memory. |
| Approach: | They propose a framework for experience-driven agent evolution that integrates procedural memory with contextual information to create a high-quality experience pool. |
| Outcome: | Experiments on BFCL-V3 and AppWorld show that ReMe outperforms memoryless Qwen3-8B. |
JEMHopQA: Dataset for Japanese Explainable Multi-Hop Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | a dataset for explainable QA in Japanese is available for many languages, but not in other languages. |
| Approach: | They present a multi-hop QA dataset based on Japanese Wikipedia . it includes question-answer pairs and supporting evidence in the form of derivation triples . they show that the dataset is sufficiently challenging for state-of-the-art LLMs based upon this dataset . |
| Outcome: | The proposed dataset is based on Japanese Wikipedia and can be used to evaluate QA tasks. |
JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for task-oriented dialogue systems in English are limited compared to Japanese. |
| Approach: | They evaluated the dialogue state tracking and response generation capabilities of Japanese language datasets using multi-domain task-oriented dialogues. |
| Outcome: | The proposed dataset provides a benchmark that is on par with MultiWOZ2.2 and the latest large language model (LLM)-based methods. |
SEER : A Knapsack approach to Exemplar Selection for In-Context HybridQA (2023.emnlp-main)
Copied to clipboard
| Challenge: | In-Context Learning with Large Language Models (LLMs) has shown great performance on reasoning tasks. |
| Approach: | They propose a method for selecting a set of exemplars that is representative and diverse. |
| Outcome: | The proposed method outperforms existing methods on FinQA and TAT-QA on hybrid questions. |
Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations (2023.emnlp-main)
Copied to clipboard
| Challenge: | open-domain chatbots focus on short single-session dialogue, neglecting the potential need for understanding contextual information in multiple consecutive sessions. |
| Approach: | They propose a 1M multi-session dialogue dataset for integrating time intervals and speaker relationships into a long-term conversation setup. |
| Outcome: | The proposed model can generate coherent responses according to time intervals and speaker relationships with high user engagement without contradiction in a long-term conversation setup. |
Open-world Multi-label Text Classification with Extremely Weak Supervision (2024.emnlp-main)
Copied to clipboard
| Challenge: | Similar single-label XWS settings cannot be easily adapted for multi-l label classification. |
| Approach: | They propose a novel method for open-world multi-label text classification under extremely weak supervision where the user provides a brief description without any labels or ground-truth label space. |
| Outcome: | The proposed method exhibits a remarkable increase in ground-truth label space coverage on various datasets. |
q2d: Turning Questions into Dialogs to Teach Models How to Search (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent dialog generation models use external search APIs to generate grounded responses. |
| Approach: | They propose an automatic data generation pipeline that generates dialogs from questions . they use a large language model to create conversational versions of question answering datasets . |
| Outcome: | The proposed method improves query generation models on a QReCC dataset. |
From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | This work sheds light on the theoretical and applied importance of dogwhistles in both NLP and computational social science. |
| Approach: | They propose a typology of dogwhistles, curate a glossary of over 300 dogwhitles and analyze their usage in historical U.S. politicians’ speeches. |
| Outcome: | The proposed model identifies dogwhistles and their meanings and shows that harmful content containing dogwhitles avoids toxicity detection. |
SIMBA UQ: Similarity-Based Aggregation for Uncertainty Quantification in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Debarun Bhattacharjya, Balaji Ganesan, Junkyu Lee, Radu Marinescu, Katya Mirylenka, Michael Glass, Xiao Shou
| Challenge: | Uncertainty quantification (UQ) provides measures of uncertainty, such as an estimate of the confidence in an LLM’s generated output. |
| Approach: | They propose a black-box approach where consistency is used as a proxy for confidence in a model's output. |
| Outcome: | The proposed methods are primarily but not necessarily entirely black- box, with consistency between output and other sampled generations used as a proxy for confidence in its correctness. |
AttributionBench: How Hard is Automatic Attribution Evaluation? (2024.findings-acl)
Copied to clipboard
| Challenge: | generative search engines enhance the reliability of large language model responses by providing cited evidence. |
| Approach: | They propose to use a benchmark to evaluate whether a large language model supports the generated responses or not . |
| Outcome: | The proposed benchmark shows that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation. |
InstOptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on automating instruction generation but do not consider other objectives that impact instruction quality. |
| Approach: | They propose an approach that treats instruction generation as an evolutionary multi-objective optimization problem. |
| Outcome: | The proposed approach improves fine-tuning performance and the generation of high-quality instructions. |
RARR: Researching and Revising What Language Models Say, Using Language Models (2023.acl-long)
Copied to clipboard
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, Kelvin Guu
| Challenge: | Language models (LMs) excel at many tasks but often produce unsupported or misleading content. |
| Approach: | They propose a system that finds attribution for any text generation model and post-edits it to fix unsupported content. |
| Outcome: | The proposed system improves attribution while preserving the original output. |
Unilaw-R1: A Large Language Model for Legal Reasoning with Reinforcement Learning and Iterative Inference (2025.emnlp-main)
Copied to clipboard
| Challenge: | Reasoning-focused large language models (LLMs) are rapidly evolving across various domains, yet their capabilities in handling complex legal problems remain underexplored. |
| Approach: | They propose a large language model tailored for legal reasoning with a 7-billion parameter scale and a two-stage training strategy combining Supervised Fine-Tuning and Reinforcement Learning. |
| Outcome: | The proposed model outperforms all models of similar scale on authoritative benchmarks and outperformed Qwen-2.5-7B-Instruct (46.6%) by an average margin of 6.6%. |
KurTail : Kurtosis-based LLM Quantization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Outliers in quantization of large language models make uniform quantization schemes less effective . a new PTQ scheme that leverages Kurtosis-based rotation mitigates outliers . |
| Approach: | They propose a new post-training quantization scheme that leverages Kurtosis-based rotation to mitigate outliers in the activations of large language models. |
| Outcome: | The proposed method outperforms existing quantization methods with 13.3% boost in MMLU accuracy and 15.5% boost in Wiki perplexity. |
Retrieval-Augmented Retrieval: Large Language Models are Strong Zero-Shot Retriever (2024.findings-acl)
Copied to clipboard
Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Yibin Lei, Tianyi Zhou, Michael Blumenstein, Daxin Jiang
| Challenge: | Large-scale retrieval is indispensable in information-seeking tasks such as open-domain question answering and knowledgegrounded dialogue. |
| Approach: | They propose to use a large language model (LLM) to augment a query with its potential answers by prompting LLMs with a composition of the query and the query’s in-domain candidates. |
| Outcome: | The proposed method breaks brute-force combinations of retrievers with LLMs and lifts the performance of zero-shot retrieval to be very competitive on benchmark datasets. |
Multi-label Sequential Sentence Classification via Large Language Model (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to sequential sentence classification are constrained by model size, sequence length, and single-label setting. |
| Approach: | They propose a large language model-based framework for both single- and multi-label SSC tasks that generate SSC labels through designed prompts. |
| Outcome: | The proposed framework enhances task understanding by incorporating demonstrations and a query to describe the prediction target. |
Mem2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation (2026.acl-long)
Copied to clipboard
Zihao Cheng, Zeming Liu, Yingyu Shan, Xinyi Wang, Xiangrong Zhu, Yunpu Ma, Hongru Wang, Yuhang Guo, Wei Lin, Yunhong Wang
| Challenge: | Existing frameworks that focus on static tools and static assets are ineffective for self-evolving agents. |
| Approach: | They propose a paradigm of co-evolutionary Capability Expansion and Experience Distillation that leverages accumulated experience to guide dynamic creation of assets. |
| Outcome: | The proposed framework improves performance in single-task and cross-task settings by 18.53% over standard LLMs, 11.80% over agents evolving solely through experience, and 6.46% over those evolving solelly through asset creation. |
Large Language Models are In-context Teachers for Knowledge Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | In-context teaching is a method of providing in-concept example rationales to a student to reason over unseen cases. |
| Approach: | They propose to use an LLM's self-elicited explanations as in-context demonstrations to prompt a student to reason over unseen cases. |
| Outcome: | The proposed model outperforms human-crafted demonstrations on medical question answering and human-created models outperfect human-made demonstrations. |
Beyond the Context Window: Scaling Agentic RL via End-to-end Optimized Context Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reinforcement learning pipelines suffer from degraded instruction following, excessive rollout costs, and strict context limits. |
| Approach: | They propose a reinforcement learning (RL) fine-tuning of large language model (LLM) agents for long-horizon multi-turn tool use where context length quickly becomes a bottleneck. |
| Outcome: | The proposed framework improves the success rate while maintaining the same or even lower working context length compared to baselines. |
Process-Supervised Reward Models for Verifying Clinical Note Generation: A Scalable Approach Guided by Domain Expertise (2025.emnlp-main)
Copied to clipboard
Hanyin Wang, Chufan Gao, Qiping Xu, Bolun Liu, Guleid Hussein, Hariprasad Reddy Korsapati, Mohamad El Labban, Kingsley Iheasirim, Mohamed Hassan, Gokhan Anil, Brian Bartlett, Jimeng Sun
| Challenge: | Currently, no automated, scalable method exists to evaluate the quality of LLM-generated clinical notes, leaving manual evaluation the gold standard. |
| Approach: | They propose a framework for training PRMs to deliver step-level reward signals for LLM-generated clinical notes. |
| Outcome: | The proposed framework outperforms reasoning and non-reasoning models on key evaluations and selects physician-preferred clinical notes with 56.2% accuracy. |
RESEMO: A Benchmark Chinese Dataset for Studying Responsive Emotion from Social Media Content (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on social media text processing do not focus on responsive emotion analysis. |
| Approach: | They propose a Chinese dataset named ResEmo for responsive emotion analysis, including 3813 posts with 68,781 comments collected from Weibo, the largest social media platform in China. |
| Outcome: | The proposed dataset includes 3813 posts with 68,781 comments collected from weibo, the largest social media platform in China. |
Amplifying Trans and Nonbinary Voices: A Community-Centred Harm Taxonomy for LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies on harms of language technology to transgender and nonbinary people focus on misgendering and stereotyping . |
| Approach: | They propose a taxonomy of harms for large language models and heuristics for evaluation to help identify harmful behavior in LLMs. |
| Outcome: | The proposed model-based approach combines surveys and focus groups with community experts to identify harmful behavior in large language models. |
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a high prompt sensitivity has been widely accepted as a core limitation of large language models . a recent study suggests that prompt senescence may be an artifact of evaluation processes . |
| Approach: | They examine whether prompt sensitivity is an inherent weakness or an artifact of evaluation . they find that heuristic evaluation methods overlook semantically correct responses . large language models have achieved remarkable success across a wide range of tasks . |
| Outcome: | The proposed model is more robust to prompt templates than previously thought . the authors show that prompt sensitivity may be an artifact of evaluation rather than a flaw . |
Multi-Attribute Steering of Language Models via Targeted Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for steering large language models fail to scale to multi-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity. |
| Approach: | They propose a steering framework for selective token-level intervention across multiple attributes that enforcing sparsity and orthogonality among vectors for different attributes. |
| Outcome: | The proposed framework outperforms existing ITI and parameter-efficient fine-tuning approaches across question answering tasks and generative tasks. |
Beyond Frameworks: Unpacking Collaboration Strategies in Multi-Agent Systems (2025.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks prioritize structural architectures and role assignments but neglect granular mechanics of agent collaboration. |
| Approach: | They propose to use centralized governance, instructor-led participation, ordered interaction patterns to optimize task accuracy and computational efficiency. |
| Outcome: | The proposed model improves task accuracy and computational efficiency under two context-dependent scenarios. |
Act2P: LLM-Driven Online Dialogue Act Classification for Power Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on explicit utterance functions, overlooking the implicit power dynamics embedded in dialogue. |
| Approach: | They propose an online Dialogue Act Classification and Dynamic Power Analysis framework based on large language models to integrate dialogue act classification with power quantification. |
| Outcome: | The proposed framework outperforms existing methods in online scenarios and shows that dialogue power is distributed and dynamically transferred. |
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multi-modal large language models typically adopt the cascade paradigm, preventing inter-modal knowledge transfer. |
| Approach: | They propose a large language model with intrinsic cross-modal conversational abilities . they construct a cross-text speech instruction dataset and employ a three-stage training strategy . |
| Outcome: | The proposed model can follow cross-modal human instructions and handle multiple modalities with one model. |
Prove Your Point!: Bringing Proof-Enhancement Principles to Argumentative Essay Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing arguments generation methods often overlook connections between opinions . Existing methods struggle with providing compelling proof . |
| Approach: | They propose a two-stage framework for argumentative essay generation with a focus on logical enhancement. |
| Outcome: | The proposed framework generates argumentative essays with better logical validity and persuasiveness than baseline models. |
New Intent Discovery with Attracting and Dispersing Prototype (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for detecting new intents with labeled data are not cluster-friendly . a robust prototypical attracting learning (RPAL) method is designed to compel instances to gravitate toward their corresponding prototype . |
| Approach: | They propose a robust and adaptive prototypical learning framework for globally distinct decision boundaries for both known and new intent categories. |
| Outcome: | The proposed method improves on CLINC, BANKING, and StackOverflow benchmarks on three challenging benchmarks. |
Weak2Wise: An Automated, Lightweight Framework for Weak-LLM-Friendly Reasoning Synthesis (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to finetuning large language models rely on expensive manual annotations or auxiliary models and fail to address the unique constraints of smaller "weak" LLMs. |
| Approach: | Weak2Wise is a fully automated framework for synthesizing highquality, weak-LLM-friendly reasoning traces. |
| Outcome: | Weak2Wise is a fully automated, lightweight framework for synthesizing highquality, weak-LLM-friendly reasoning traces. |
I²B-LPO: Latent Policy Optimization via Iterative Information Bottleneck (2026.acl-long)
Copied to clipboard
Huilin Deng, Hongchen Luo, Yue Zhu, Long Li, Zhuoyue Chen, Xinghao Zhao, Ming LI, Chuyang Zhao, Jihai Zhang, MengChang Wang, Yang Cao, Yu Kang
| Challenge: | Existing methods for large language model reasoning suffer from exploration collapse due to the semantic homogeneity of random rollouts. |
| Approach: | They propose to use latent policy optimization via iterative information bottleneck to optimize reasoning trajectories by diversifying reasoning . |
| Outcome: | Empirical results show that the proposed method achieves state-of-the-art performance with margins of up to 5.3% in accuracy and 7.4% in diversity metrics. |
Diversity-oriented Data Augmentation with Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing data augmentation methods focus on increasing sample numbers while neglecting sample distribution diversity, which can lead to model overfitting. |
| Approach: | They propose a data augmentation framework that focuses on sample distribution diversity and trains a large language model as a diverse paraphraser. |
| Outcome: | The proposed framework achieves an average performance gain of 10.52% surpassing the runner-up baseline with more than three percentage points. |
Right at My Level: A Unified Multilingual Framework for Proficiency-Aware Text Simplification (2026.acl-long)
Copied to clipboard
| Challenge: | Existing large language model-based readability control methods rely on pre-labeled sentence corpora and primarily target English. |
| Approach: | They propose a framework for adaptive multilingual text simplification without parallel corpora supervision that integrates three reward modules: vocabulary coverage, semantic preservation, and coherence. |
| Outcome: | The proposed framework achieves higher lexical coverage at target proficiency levels while maintaining original meaning and fluency compared to stronger LLMs. |
CREPE: Rapid Chest X-ray Report Evaluation by Predicting Multi-category Error Counts (2025.emnlp-main)
Copied to clipboard
| Challenge: | CREPE is a metric for rapid, interpretable, and clinically grounded automated chest X-ray report generation. |
| Approach: | They propose to use a domain-specific BERT model fine-tuned with a multi-head regression architecture to predict multi-category error counts across six clinically meaningful categories. |
| Outcome: | CREPE outperforms traditional and recent metrics on a large-scale synthetic dataset of 32,000 annotated report pairs. |
WET: Overcoming Paraphrasing Vulnerabilities in Embeddings-as-a-Service with Linear Transformation Watermarks (2025.acl-long)
Copied to clipboard
| Challenge: | Existing EaaS watermarks can be removed by paraphrasing when attackers clone the model. |
| Approach: | They propose a method that integrates a target embedding into the original embeddable based on the presence of trigger words in the input text. |
| Outcome: | The proposed technique is empirically and theoretically robust against paraphrasing. |
HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing positional encodings exhibit long-term decay, based on an entrenched and long-standing opinion that tokens farther away from the current position carry less relevant information. |
| Approach: | They propose a high-frequency rotary position encoding (HoPE) that replaces specific components in RoPE with position-independent ones, retaining only high- frequency signals. |
| Outcome: | The proposed method exhibits greater robustness to the out-of-distribution behavior in attention patterns during extrapolation. |
Improving Occupational ISCO Classification of Multilingual Swiss Job Postings with LLM-Refined Training Data (2025.findings-acl)
Copied to clipboard
| Challenge: | 80% of job postings are German, 11% French, 8% English, and under 1% Italian. |
| Approach: | They propose a method that refines silver-standard ISCO labels by consolidating them with predictions from pre-fine-tuned models to resolve discrepancies. |
| Outcome: | The proposed method raises Top-1 accuracy on silver data to 58.3% and reaches 80% precision on held-out data. |
Multi-expert Prompting Improves Reliability, Safety and Usefulness of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing enhancements of ExpertPrompting improve the large language model generation process. |
| Approach: | They propose a novel enhancement of ExpertPrompting to improve LLM generation by simulating multiple experts, aggregating their responses and selecting the best among individual and aggregated responses. |
| Outcome: | The proposed enhancement outperforms ExpertPrompting and comparable baselines in truthfulness, factuality, informativeness, usefulness and harmfulness. |
MS-RAG: Simple and Effective Multi-Semantic Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for large language models suffer from poor indexing and inference speed . graph-based RAGs heavily rely on LLM for retrieval thus inference slow . |
| Approach: | They propose retrieval-augmented generation (RAG) which integrates knowledge with dense vectors to build a multi-semantic RAG. |
| Outcome: | The proposed method achieves state-of-the-art performance with faster inference speed compared to existing methods . |
FastDraft: How to Train Your Draft (2025.findings-acl)
Copied to clipboard
| Challenge: | Speculative Decoding relies on the availability of efficient draft models, which are often lacking due to a stringent constraint of vocabulary compatibility. |
| Approach: | They propose a novel approach for pre-training and aligning a draft model to any large language model by incorporating efficient pre-train and fine-tuning over synthetic datasets generated by the target model. |
| Outcome: | The proposed model can be trained on a single server with 8 Intel Gaudi 2 accelerators in under 24 hours and achieves 3x acceptance rate, block efficiency and 2x memory bound speedup. |
A Decoupled Multi-Agent Framework for Complex Text Style Transfer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for text style transfer struggle with complex styles . existing models perform well on simple styles like sentiment and formality . |
| Approach: | They propose a multi-agent self-check framework that includes a large language model as a planner for disentangling subtasks and expert agents for executing the subtask. |
| Outcome: | The proposed framework significantly improves style strength and content preservation on simple and complex style datasets. |
Mamba Drafters for Speculative Decoding (2025.findings-emnlp)
Copied to clipboard
Daewon Choi, Seunghyuk Oh, Saket Dingliwal, Jihoon Tack, Kyuyoung Kim, Woomin Song, Seojin Kim, Insu Han, Jinwoo Shin, Aram Galstyan, Shubham Katiyar, Sravan Babu Bodapati
| Challenge: | Existing drafters that use external drafters suffer from slower drafting while self-speculation methods use drafters tailored to the target model but require re-training. |
| Approach: | They propose a drafter based on a state space model, Mamba, as a solution that combines the best aspects of both approaches. |
| Outcome: | The proposed drafters outperform existing drafters while using less memory and maintaining their cross-model adaptability. |
Quite Good, but Not Enough: Nationality Bias in Large Language Models - a Case Study of ChatGPT (2024.lrec-main)
Copied to clipboard
| Challenge: | Nationality is a key demographic element that enhances the performance of large language models, but it has received less scrutiny regarding inherent biases. |
| Approach: | They investigated nationality bias in ChatGPT, a large language model for text generation. |
| Outcome: | The proposed model generates 4,680 discourses about nationality in Chinese and English, with 195 countries, 4 temperature settings, and 3 prompt types. |
ZERA: Zero-init Instruction Evolving Refinement Agent – From Zero Instructions to Structured Prompts via Principle-based Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve large language model performance focus on user prompts and require large sample sizes and long iteration cycles. |
| Approach: | They propose a framework that jointly optimizes both system and user prompts . they evaluate ZERA across five LLMs and nine diverse datasets spanning reasoning, summarization, and code generation tasks. |
| Outcome: | The proposed framework improves prompt construction over baselines and is available on github . it scores prompts using eight generalizable criteria and revises prompts based on structured critiques. |
SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities (2025.findings-acl)
Copied to clipboard
Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, Radha Poovendran
| Challenge: | Emerging large reasoning models (LRMs) leverage long chain-of-thought (CoT) reasoning to enhance their reasoning capabilities. |
| Approach: | They conduct a systematic study of LRM safety using human annotations to assess their safety. |
| Outcome: | The proposed safety measures are compared to state-of-the-art models on strong and wildjailbreak datasets. |
ETRQA: A Comprehensive Benchmark for Evaluating Event Temporal Reasoning Abilities of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Event temporal reasoning (ETR) is a significant indicator that a large language model understands the physical world. |
| Approach: | They propose a unified taxonomy for event temporal questions and construct a benchmark based on this taxonomies. |
| Outcome: | The proposed taxonomy inherits and expands existing datasets and contains multiple categories of compound questions. |
Task Facet Learning: A Structured Approach To Prompt Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to prompt optimization are limited to learning multiple facets of a task from training examples. |
| Approach: | They propose to optimize a text prompt by considering different facets of a task and including them in the prompt. |
| Outcome: | The proposed algorithm can generate long, complex prompts that existing methods are unable to generate. |
Topic-Based Watermarks for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing watermarking methods often involve trade-offs between attack robustness, generation quality and additional overhead. |
| Approach: | They propose a topic-guided watermarking scheme that partitions the vocabulary into topic-aligned token subsets. |
| Outcome: | The proposed method achieves text quality comparable to industry-leading systems and improves watermark robustness against paraphrasing and lexical perturbation attacks with minimal performance overhead. |
MERIT: Multi-Agent Collaboration for Unsupervised Time Series Representation Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to time series representation learning are time-consuming and expert-dependent, which are difficult to generalize across different tasks. |
| Approach: | They propose to use large language model agent to guide unsupervised time series representation learning and a framework to integrate three LLM agents to collaboratively generate positive views for time series data. |
| Outcome: | The proposed framework integrates large language model (LLM) agent to guide unsupervised time series representation learning and compares it with state-of-the-art baselines on multiple time series datasets. |
Human or LLM as Standardized Patients? A Comparative Study in Medical Education (2026.acl-long)
Copied to clipboard
| Challenge: | Standardized patients (VSPs) are indispensable for clinical skills training but remain expensive and difficult to scale. |
| Approach: | They propose a multi-agent VSP framework that separates case-grounded information disclosure from response generation to support stable, inquiry-conditioned patient behavior. |
| Outcome: | The proposed framework more closely matches human SP behavior than existing VSPs, particularly in case consistency and controlled disclosure. |
EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records (2024.emnlp-main)
Copied to clipboard
Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce Ho, Carl Yang, May Dongmei Wang
| Challenge: | EHRAgent enables clinicians to interact with EHRs using natural language . reliance on rule-based conversion systems often necessitates additional training or effort from data engineers. |
| Approach: | They propose a large language model agent that generates and executes code in natural language to facilitate clinicians in directly interacting with EHRs. |
| Outcome: | The proposed agent outperforms the strongest baseline by up to 29.6% in success rate on three real-world EHR datasets. |
Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have not examined how backdoored agents can influence tool-use sequences to perform harmful actions. |
| Approach: | They propose a backdoor attack framework that embeds semantic triggers into fine-tuned LLM agents. |
| Outcome: | The proposed framework embeds semantic triggers into fine-tuned LLM agents . when triggered, the backdoored agent invokes memory-access tool calls to retrieve stored user context and exfiltrates it via disguised retrieval tool calls. |
Select High-quality Synthetic QA Pairs to Augment Training Data in MRC under the Reward Guidance of Generative Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches focus on downstream metrics to select QA pairs, which lack generalization across different datasets. |
| Approach: | They propose a general selection method that uses a large pre-trained language model as a reward model in a Reinforcement Learning framework for the training of the selection agent. |
| Outcome: | The proposed method improves performance on generative and extractive datasets. |
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026.findings-acl)
Copied to clipboard
Wentao Shi, Yu Wang, Yuyang Zhao, Yuxin Chen, Fuli Feng, Xueyuan Hao, Xi Su, Qi GU, Hui Su, Xunliang Cai, Xiangnan He
| Challenge: | Existing approaches to verify agent behaviors in complex environments rely on rule-based verifiers or LLM-as-a-Judge models. |
| Approach: | They propose a benchmark to evaluate Agent-as-a-Judge across three domains . the benchmark covers search, data systems, and graphical user interfaces - with 155 tasks and 516 trajectories . |
| Outcome: | The proposed benchmark outperforms existing benchmarks in search, data systems, and GUI domains while revealing open challenges in agent-based verification. |
SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Traditionally, estimating item difficulties requires real students to respond to items . a cold-start approach cannot be applied to previously unseen items either . |
| Approach: | They propose a method for aligning simulated students with instructed ability to predict difficulty of open-ended items. |
| Outcome: | The proposed method outperforms existing methods on two real-world student responses. |
Sequence-to-Sequence Language Models for Character and Emotion Detection in Dream Narratives (2024.lrec-main)
Copied to clipboard
| Challenge: | Sigmund Freud's interpretation of dreams has been central to understanding human consciousness for centuries. |
| Approach: | They propose to automate the annotation process by using a natural language framework . they evaluate the impact of model size, prediction order of characters, and consideration of proper names and character traits . |
| Outcome: | The proposed model performs better with a large language model while having 28 times fewer parameters. |
D2CS - Documents Graph Clustering using LLM supervision (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Document clustering does not inherently ensure thematic consistency. |
| Approach: | They propose a framework that constructs a similarity graph over document embeddings and applies iterative graph-based clustering algorithms to partition the corpus into initial clusters. |
| Outcome: | The proposed framework constructs a similarity graph over document embeddings and applies iterative graph-based clustering algorithms to partition the corpus into initial clusters. |
Do Generalisation Results Generalise? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies evaluating generalisation performance on large language models focuses on a single out-of-distribution dataset . |
| Approach: | They examine whether OOD generalisation results generalise across multiple OOD testsets throughout a finetuning run and then evaluate the partial correlation of results . |
| Outcome: | The proposed model achieves high scores on multiple OOD testsets, regressing out in-domain performance. |
BIG-Bench Extra Hard (2025.acl-long)
Copied to clipboard
Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, Nishanth Dikkala, Gladys Tyen, Xin Liu, Uri Shalit, Silvia Chiappa, Kate Olszewska, Yi Tay, Vinh Q. Tran, Quoc V Le, Orhan Firat
| Challenge: | Current benchmarks for large language model reasoning focus on math and coding abilities, leaving a gap in evaluating broader reasoning proficiencies. |
| Approach: | They propose a benchmark to evaluate general reasoning in large language models . they use BIG-Bench and its harder version BIG-Benefit Hard to assess general reasoning . |
| Outcome: | The new benchmark pushes the boundaries of LLM reasoning evaluation. |
Towards Fine-grained Audio Captioning with Multimodal Contextual Fusion (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for audio captioning lack fine-grained detail and contextual accuracy due to limited unimodal or superficial information. |
| Approach: | They propose a two-stage automated pipeline that uses pretrained models to extract contextual cues from video . a large language model synthesizes these inputs to generate detailed and context-aware captions . |
| Outcome: | The proposed method is scalable and generates detailed and context-aware captions on large-scale audio datasets. |
Third-Person Appraisal Agent: Simulating Human Emotional Reasoning in Text with Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Emotional reasoning is essential for improving human-AI interactions, especially in mental health support and empathetic systems. |
| Approach: | They propose a third-person appraisal agent that simulates human-like emotional reasoning through three phases: Primary Appraisal, Secondary Appraisals, and Reappraisal. |
| Outcome: | The proposed model outperforms baseline LLMs in various emotional reasoning tasks, demonstrating superior generalization and interpretability. |
“Give Me BF16 or Give Me Death”? Accuracy-Performance Trade-Offs in LLM Quantization (2025.acl-long)
Copied to clipboard
| Challenge: | despite popularity of large language model quantization, there are significant accuracy-performance trade-offs associated with quantization formats. |
| Approach: | They evaluate popular quantization formats across academic benchmarks and real-world tasks . they also examine the difference in text generated by quantized models versus their uncompressed counterparts . |
| Outcome: | The proposed format is lossless across all model scales and incurs low accuracy degradation when properly tuned. |
Reverse Prompt Engineering: A Zero-Shot, Genetic Algorithm Approach to Language Model Inversion (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods rely on a large number of outputs for training and inference, and they can produce garbled text. |
| Approach: | They propose a training-free framework that reconstructs prompts using only a limited number of text outputs from a language model. |
| Outcome: | The proposed framework generates high-quality prompt recovery and more semantically and functionally aligned with the originals than current state-of-the-art methods. |
QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing video-to-text summarization evaluation methods depend heavily on human-written reference summaries. |
| Approach: | They propose a reference-free metric evaluating candidate summaries directly against source videos through multimodal question answering. |
| Outcome: | The proposed metric assesses candidate summaries directly against source videos through multimodal question answering. |
A Constrained Text Revision Agent via Iterative Planning and Searching (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing text revision systems are capable of generating fluent and coherent text, but struggle with constrained text revision (CTR). |
| Approach: | They propose a tool that generates revisions tailored to different scenarios using a planner, a reviser and adaptable tools. |
| Outcome: | The proposed agent outperforms baseline approaches in both constraint adherence and revision quality. |
Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools (2025.acl-long)
Copied to clipboard
| Challenge: | Existing reasoning methods excel in structured domains like math and code, but they are not all effective in knowledge-intensive tasks. |
| Approach: | They introduce a framework that enhances large language model reasoning by integrating external tool-using agents. |
| Outcome: | The proposed framework achieves state-of-the-art among public models and delivers comparable performance to OpenAI Deep Research. |
Cache Saver: A Modular Framework for Efficient, Affordable, and Reproducible LLM Inference (2025.findings-emnlp)
Copied to clipboard
Nearchos Potamitis, Lars Henning Klein, Bardia Mohammadi, Chongyang Xu, Attreyee Mukherjee, Niket Tandon, Laurent Bindschaedler, Akhil Arora
| Challenge: | Inference is the major cost throughout the lifecycle of a large language model (LLM). |
| Approach: | They propose a plug-and-play, asynchronous framework that facilitates high-level inference optimizations. |
| Outcome: | The proposed framework reduces cost and CO2 by 35% across methods, tasks, and LLMs. |
Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to reasoning faithfulness violate constraints, authors say . a science fantasy series and companion books are among the books . |
| Approach: | They propose a framework that enforces verification over internal belief states within the agent before action commitment, achieving faithful reasoning. |
| Outcome: | The proposed framework improves reasoning faithfulness while preserving competitive end-task performance. |
Explaining Differences Between Model Pairs in Natural Language through Sample Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | a framework that generates faithful natural language explanations of when and how two ML models converge or diverge in their predictions requires access to training data. |
| Approach: | They propose a framework that generates faithful natural language explanations of when and how two ML models converge or diverge in their predictions. |
| Outcome: | The proposed framework generates faithful natural language explanations of when and how two models diverge in their predictions. |
Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for factuality checking require dataset-specific threshold tuning, while LLM-based approaches often use direct prompting. |
| Approach: | They propose to use a reading comprehension task to check for true/false claim factuality and prompt LLMs with explicit test-taking strategies for efficient reasoning. |
| Outcome: | The proposed method reduces token usage by over 80% compared to unguided open-ended reasoning and achieves competitive performance to more expensive alternatives. |
InteracSPARQL : An Interactive System for SPARQL Query Refinement Using Natural Language Explanations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for SPARQL generation rely on one-turn models. |
| Approach: | They propose a training-free interactive refinement pipeline that acts as a plug-and-play enhancement for existing SPARQL systems. |
| Outcome: | The proposed approach improves the accuracy of base models without fine-tuning . it transforms potentially flawed queries from any source into verifiable code . |
TestNUC: Enhancing Test-Time Computing Approaches and Scaling through Neighboring Unlabeled Data Consistency (2025.acl-long)
Copied to clipboard
Henry Peng Zou, Zhengyao Gu, Yue Zhou, Yankai Chen, Weizhi Zhang, Liancheng Fang, Yibo Wang, Yangning Li, Kay Liu, Philip S. Yu
| Challenge: | Test-time computing approaches that leverage additional computational resources during inference have been proven effective in enhancing large language model performance. |
| Approach: | They propose a linearly scaling approach that leverages local consistency of neighboring unlabeled data to improve test-time predictions. |
| Outcome: | The proposed approach outperforms baseline methods such as prompting and self-consistency across eight datasets and performs robustly across embedding models. |
CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory (2025.acl-long)
Copied to clipboard
Weichen Zhang, Chen Gao, Shiquan Yu, Ruiying Peng, Baining Zhao, Qian Zhang, Jinqiang Cui, Xinlei Chen, Yong Li
| Challenge: | Existing ground VLN agents struggle in aerial VLLN due to the lack of predefined navigation graphs and the exponentially expanding action space in long-horizon exploration. |
| Approach: | They propose a large language model-empowered aerial VLN agent that decomposes the long-horizon task into sub-goals with different semantic levels. |
| Outcome: | The proposed method achieves state-of-the-art performance with significant improvement in continuous city environments. |
SRA: Span Representation Alignment for Large Language Model Distillation (2026.acl-long)
Copied to clipboard
Quoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi, Tung Nguyen, Linh Ngo Van, Nguyen Thi Ngoc Diep, Trung Le
| Challenge: | Existing approaches to cross-tokenizer knowledge distillation are brittle and sensitive to discrepancies between tokenizers. |
| Approach: | They propose a framework that shifts the unit of alignment from tokens to robust, tokenizer-agnostic spans and employ a geometric regularizer to preserve the structural integrity of the representation space. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in cross-architecture distillation experiments. |
WikiFactDiff: A Large, Realistic, and Temporally Adaptable Dataset for Atomic Factual Knowledge Update in Causal Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Factual update is a task of inserting, replacing, or removing facts in large language models. |
| Approach: | They present a dataset that describes the evolution of factual knowledge between two dates as a collection of simple facts divided into three categories: new, obsolete, and static. |
| Outcome: | The proposed dataset compares the state of the Wikidata knowledge base at 4 January 2021 and 27 February 2023. |
Datamart-Agent: LLM-Driven Game-Theoretic Agent for Data Marketplace Modeling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on data marketplaces model static equilibria and complete information, which limits their realism. |
| Approach: | They propose an LLM-driven game-theoretic agent that makes equilibrium-consistent decisions in analytically tractable data marketplaces with evolving and incomplete-information. |
| Outcome: | The proposed framework matches equilibrium-consistent decision execution in a static data marketplace with a dynamic game tree memory and mechanism-guided reflection without updating parameters. |
Multi-Drafter Speculative Decoding with Alignment Feedback (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to accelerate large language model (LLM) inference use a smaller model to draft future tokens, which are then verified by the target LLM. |
| Approach: | They propose a unified framework that integrates multiple drafters into the SD process. |
| Outcome: | Extensive experiments show that MetaSD outperforms single-drafter approaches. |
microCLIP: Unsupervised CLIP Adaptation via Coarse-Fine Token Fusion for Fine-Grained Image Classification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing UA methods for fine-grained image classification rely on coarse-grain visual tokens, which misses fine spatial details. |
| Approach: | They propose a label-free self-training framework that adapts visual features and LLMderived text prototypes using fine-grained cues. |
| Outcome: | The proposed framework improves alignment between finegrained visual regions and rich textual descriptions while updating only layer norms and a tiny head. |
SpecMind: Cognitively Inspired, Interactive Multi-Turn Framework for Postcondition Inference (2026.acl-long)
Copied to clipboard
Cuong Chi Le, Minh V.t. Pham, Tung D. Vu, Van Duc Cuong, Phan Nhat Huy, Phan Nhat Hoang, Tien N. Nguyen
| Challenge: | Existing methods for generating specifications are limited and often fail to infer semantic specifications such as pre-/postconditions. |
| Approach: | They propose a framework that treats LLMs as exploratory reasoners rather than one-shot generators. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in accuracy and completeness of generated postconditions. |
Demystifying Multi-Agent Debate: The Role of Confidence and Diversity (2026.findings-acl)
Copied to clipboard
| Challenge: | Multi-agent debate (MAD) is widely used to improve large language models' (LLMs) reasoning and test-time scaling. |
| Approach: | They propose a diversity-aware initialisation that selects a more diverse pool of candidate answers, increasing the likelihood that a correct hypothesis is present at the start of debate. |
| Outcome: | The proposed protocol outperforms vanilla MAD and majority vote on six reasoning-oriented QA benchmarks. |
Joint Knowledge Base Completion and Question Answering by Combining Large Language Models and Small Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies rely on the small language model (SLM) to enhance them jointly, and the large language model’s strong reasoning ability is ignored. |
| Approach: | They propose a framework which can make knowledge base completion and knowledge base question answering enhance each other in an iterative manner by combining the strengths of the small language model and the large language model. |
| Outcome: | The proposed framework surpasses baselines for both KBC and KBQA tasks over two public benchmark data sets. |
MPCG: Multi-Round Persona-Conditioned Generation for Modeling the Evolution of Misinformation with LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Misinformation evolves as it spreads, shifting in language, framing, and moral emphasis to adapt to new audiences. |
| Approach: | They propose a multi-round, persona-conditioned framework that simulates how claims are iteratively reinterpreted by agents with distinct ideological perspectives. |
| Outcome: | The proposed framework generates persona-specific claims across multiple rounds . it is based on an uncensored large language model and is scalable to multiple tasks . |
Introducing Spotlight: A Novel Approach for Generating Captivating Key Information from Documents (2025.emnlp-main)
Copied to clipboard
Ankan Mullick, Sombit Bose, Rounak Saha, Ayan Kumar Bhowmick, Aditya Vempaty, Prasenjit Dey, Ravi Kokku, Pawan Goyal, Niloy Ganguly
| Challenge: | Unlike highlights (fragmented key points) and traditional summaries, spotlights selectively emphasize intriguing content to foster deeper reader engagement with the source material. |
| Approach: | They propose a novel paradigm for information extraction that selectively emphasizes intriguing content to foster deeper reader engagement with the source material. |
| Outcome: | The proposed model improves readability and boosts engagement value of the original document. |
Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models (2026.findings-acl)
Copied to clipboard
Bang Zhang, Ruotian Ma, Qingxuan Jiang, Peisong Wang, Jiaqi Chen, Zheng Xie, Xingyu Chen, Yue Wang, Fanghua Ye, Jian Li, Yifan Yang, Zhaopeng Tu, Xiaolong Li
| Challenge: | Large language models (LLMs) have evolved from statistical sequence predictors to sophisticated autonomous agents capable of reasoning, planning, and sustaining multi-turn conversa-tions. |
| Approach: | They propose a system that instantiates a "Sentient Agent" that simulates human-like emotional changes and inner thoughts to provide a more realistic evaluation of the model in multi-turn conversations. |
| Outcome: | The proposed framework measures the agent's higher-order social cognition in multi-turn conversations. |
Reasoning Up the Instruction Ladder for Controllable Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Current models struggle to balance competing directives, causing conflicting instructions. |
| Approach: | They propose to reframe instruction hierarchy resolution as a reasoning task . they use a training dataset to enable this capability by transferring general reasoning capabilities to instruction prioritization . |
| Outcome: | The proposed method improves on safety-critical scenarios beyond the training distribution and jailbreaks. |
SAM-NER: Semantic Archetype Mediation for Zero-Shot Named Entity Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | Zero-shot Named Entity Recognition (ZS-NER) remains brittle under domain and schema shifts, where unseen label definitions misalign with a large language model’s intrinsic semantic organization. |
| Approach: | They propose a framework that stabilizes cross-domain transfer through an intermediate, domain-invariant archetype space. |
| Outcome: | Experiments on the CrossNER benchmark show that SAM-NER consistently outperforms strong prior ZS-NER baselines in cross-domain settings. |
Toward Robust Evaluation for Multilingual Grammatical Error Correction: Can Large Language Models Replace Human References? (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has shown that using aclosest-gold reference yields more accurate performance estimates, but producing such references for each system individually is costly. |
| Approach: | They propose a method for generating closest-gold references by prompting a large language model with system outputs and a standard reference-based evaluations show weak or no correlation. |
| Outcome: | The proposed method outperforms state-of-the-art models on 14 languages across 14 benchmarks. |